feat: Switch compaction model from session model - #3
Open
ryangamerdev wants to merge 23 commits into
Open
Conversation
ryangamerdev
force-pushed
the
feature/compaction-model
branch
3 times, most recently
from
March 1, 2026 03:14
00b3b7d to
80b6e35
Compare
ryangamerdev
force-pushed
the
feature/compaction-model
branch
from
March 9, 2026 02:30
80b6e35 to
8756834
Compare
Collapse compaction mode: - Selectively compresses oldest 65% of tokens instead of entire conversation - Merges historical summaries for continuity (configurable: previousSummaries) - Places summary at correct breakpoint position in timeline - TUI toggle cycles standard -> collapse -> float via command palette - insertTriggers=false prevents timestamp collision infinite loops - Preserves real user messages; only deletes synthetic trigger messages Float compaction mode: - Sub-collapses oldest conversation chains before overflow evaluation - Chain detection requires 2+ assistant messages (skips simple Q&A pairs) - bookend algorithm by default; configurable per chainThreshold - Soft-deletes sub-collapsed messages with flux=compacted (not hard-delete) - summary:true flag on sub-collapse result prevents re-trigger loops - detectChains skips already-processed messages (summary:true or flux set) to prevent infinite sub-collapse loop - Token adjustment accounts for sub-collapse savings to prevent re-trigger - Reloads TUI messages after sub-collapse via session.compacted event - Re-parents orphaned chain messages after mid-chain split - splitChainMinThreshold gate prevents processing chains too small for meaningful sub-collapse Sub-collapse prompt: - Extracts only information lost if messages were deleted (not replacement) - Anti-repetition: user request and earlier summaries marked REFERENCE ONLY - Structured extraction: final artifacts, critical determinations, non-obvious discoveries, final state - Explicit discard list: intermediate attempts, debugging steps, narration - Produces factual statements, not narrative or conversational response Knowledge pack support: - KPs stored as flux:knowledge user messages at time_created=1,2,... - Loaded from ~/.config/opencode/llm_knowledge_packs/ as .yaml/.yml via Bun.YAML - Keyed by name@version throughout; idempotent inject on every message send - filterCompacted skips flux=knowledge messages to prevent duplication - KP messages prepended explicitly via KnowledgePack.fromSession() before filterCompacted result (KPs sit before compaction breakpoint so filterCompacted never returns them) - toModelMessages prepends KPs first, then === USER MESSAGE === delimiter, then skips flux-tagged messages in normal loop - Trailing newline on delimiter prevents concatenation with first user message - Sidebar KP section: collapsed view shows active packs, expanded shows full library with click-to-toggle enable/disable - Sidebar reactively updates via kpMessageCount memo on sync store changes - Dedicated API endpoints: GET/POST/DELETE /:sessionID/knowledge-packs Config options added: compaction.method: standard | collapse | float compaction.trigger: overflow threshold (default 0.85) compaction.extractRatio: fraction to extract (default 0.65) compaction.recentRatio: recent context reference (default 0.15) compaction.summaryMaxTokens: target summary size (default 10000) compaction.previousSummaries: history to merge (default 3) compaction.insertTriggers: whether to create trigger messages (default false for collapse/float) compaction.float.chainThreshold: chains to maintain (default 3) compaction.float.algorithm: bookend | full | minimal (default bookend) compaction.float.subCollapseSummaryMaxTokens: target tokens for sub-collapse
Adds a `minFloat` parameter (default 0.6) that gates sub-collapse
evaluation in float compaction mode based on actual context window usage.
Two guard points are added:
1. Pre-check gate: before any chain detection, the initial token count
(input + cache.read + cache.write + output from the last finished
assistant message) is compared against the context limit. If usage is
below minFloat the entire sub-collapse path is skipped and the
function returns early with subCollapsed: false.
2. Per-chain gate: sub-collapse chains are processed one at a time via
a recursive collapseNext() function. After each successful collapse,
messages are reloaded and token usage is re-estimated via
estimateMessageTokens(). If the estimated usage fraction drops below
minFloat before the next chain is evaluated, recursion stops. This
means five chains above threshold may result in only one or two
collapses if the first brings usage below minFloat.
The recursive design replaces the previous single-chain execution and
avoids let/mutation in favour of the immutable patterns preferred by
the style guide.
Config schema adds minFloat to compaction.float in config.ts with a
0..1 range validator. Can be overridden in opencode.json:
{ "compaction": { "float": { "minFloat": 0.5 } } }
The call site in prompt.ts now passes tokens and contextLimit to
floatModePreCheck so the real API-reported token counts are used for
the initial check rather than an estimate.
Previously, after each sub-collapse in collapseNext(), the token count for the next minFloat evaluation was re-estimated by summing estimateMessageTokens() across the entire reloaded message list. This was flawed in two ways: 1. estimateMessageTokens() only counts text parts and completed tool parts. It silently ignores reasoning parts, step-start markers, compaction parts, and system/cache overhead — so the re-estimate was always lower than the real context usage, making the minFloat gate think context had dropped further than it actually had. 2. Because the re-estimate was consistently too low, the gate would pass on every iteration and sub-collapse would continue collapsing all qualifying chains in sequence regardless of whether the context had actually dropped below minFloat — defeating the purpose of the per-chain check entirely. Fix: replace the full re-estimation with a delta calculation: nextTokenCount = tokenCount - chain.chainTokens + result.summaryTokens - chain.chainTokens is the token cost of the collapsed chain as measured by detectChains() using estimateMessageTokens() scoped to chain messages only. Using the same estimator for both the baseline and the deduction keeps the accounting consistent — any systematic under/over-count cancels out. - result.summaryTokens is the actual output token count reported by the model for the summary it generated, so the added-back tokens are precise. The reloaded message list is still fetched after each collapse so that shouldFloatSubCollapse() -> detectChains() sees the updated conversation state for chain detection. Only the token count tracking is changed — it now uses the running delta rather than a full re-scan. Added a log.info() line after each collapse emitting tokensBefore, chainTokensRemoved, summaryTokensAdded, tokensAfter, and usedFractionAfter so the accounting can be verified during testing.
The minFloat gate in collapseNext was comparing against stale token counts after the first chain collapse because the delta reduction only existed in memory — it was never written back to the database. Root cause: process() (the main collapse compaction) works by finding the chronologically last real finished assistant message after each collapse and patching its stored tokens field directly in the database, stuffing the post-collapse total into tokens.cache.read. This is what isOverflow() and the prompt loop read on every iteration via lastFinished.tokens. collapseNext was only tracking the reduction in memory via nextTokenCount. On the second and subsequent collapseNext iterations, lastFinished.tokens was read fresh from the database and still held the original pre-collapse values, so the minFloat check always saw the same high initial usage and never stopped. Fix: after each successful executeSubCollapse, load all session messages, find the last real finished assistant message (excluding the new summary by its ID, and requiring finish to be set), and patch its stored token counts using the same formula process() uses: newTotal = currentTotal - chain.chainTokens + summaryTokens tokens.input = 0 tokens.cache.read = max(0, newTotal - tokens.output) tokens.cache.write = 0 tokens.output = unchanged tokens.reasoning = unchanged Writing via Session.updateMessage() persists the reduction to SQLite and fires message.updated to the TUI, so the sidebar context percentage now updates after each chain collapse rather than only at the end. The nextTokenCount delta is kept for the immediate recursive minFloat check within the same collapseNext call since the database write is async and the next iteration reads lastFinished fresh anyway.
When a subagent session is created via the Task tool (e.g. explore,
general, or any user-defined subagent), its context window starts empty.
The existing KP injection in loop() only auto-injects packs from the
filesystem default dir and config-declared packs — it has no visibility
into knowledge packs that were manually enabled in the parent session
via the sidebar (flux:knowledge messages stored in the parent session DB).
This change ensures subagent sessions inherit all active knowledge packs
from their parent session, giving them the same knowledge context the
parent LLM has.
Changes:
knowledge-pack.ts — add KnowledgePack.copyFromParent()
Reads the parent session's active flux:knowledge messages via the
existing fromSession() helper and writes them verbatim into the child
session using the same Session.updateMessage / Session.updatePart
pattern used by inject() and add(). Fully idempotent: deduplicates
by agent key (kp:<name>@<version>) so packs already present in the
child (e.g. auto-injected from defaultDir) are not duplicated. The
rendered [KNOWLEDGE PACK: ...] content is copied exactly so the
subagent sees the same formatted text as the parent.
prompt.ts — call copyFromParent() in loop() for child sessions
Added after the existing KP injection block (defaultDir inject +
config packs auto-enable) so all three sources are applied in order:
1. inject() — filesystem packs from defaultDir / config paths
2. configPacks — config-declared packs with enabled: true
3. copyFromParent — manually sidebar-enabled parent session packs
Gated on session.parentID so it only runs for child sessions and is
a no-op for top-level sessions (no extra DB reads in the normal path).
Covers all subagent types (explore, general, user-defined mode:subagent
and mode:all agents) because they all go through the same Task tool ->
Session.create -> SessionPrompt.prompt -> loop() code path.
Knowledge packs can now declare per-agent system prompt overrides in
their YAML under the `agent` field:
agent:
explore:
prompt: |-
You are a coder tool specialist...
When a pack containing agent overrides is active in a session (either
via config `knowledge.packs[].enabled: true` or enabled from the
sidebar), the matching agent's built-in system prompt is replaced with
the KP-supplied prompt for the duration of that session.
Both enablement paths produce identical flux:knowledge messages in the
session DB, which is the single source of truth agentPrompts() reads.
The global Agent registry is never mutated — the override is applied
per loop iteration as a shallow clone of the Agent.Info object.
Changes:
packages/opencode/src/session/knowledge-pack.ts
- KPFile internal type: add `agent?: Record<string, { prompt?: string }>`
field so the YAML parser captures agent overrides at load time
- Pack exported type: add matching `agent` field so callers can
inspect overrides after load()
- load(): forward kp.agent into the Pack object alongside existing fields
- agentPrompts(sessionID): new exported function that
1. reads all flux:knowledge messages from the session to determine
which packs are active (strips the "kp:" prefix to get name@version)
2. loads pack files from both defaultDir() and libraryDir() so it
covers auto-injected packs (kp/) and sidebar-enabled packs
(llm_knowledge_packs/) in one pass
3. returns a merged Record<agentName, prompt> where later packs in
filesystem order win if multiple packs override the same agent
packages/opencode/src/session/prompt.ts
- loop(): resolve agentBase via Agent.get() then call
KnowledgePack.agentPrompts(sessionID); if the resolved agent name has
a KP override, shadow agentBase with a spread that replaces only the
prompt field before passing agent into the rest of the loop iteration
(insertReminders, resolveTools, processor.process)
After any onMouseDown in the sidebar, opentui clears
currentFocusedRenderable to null because sidebar box elements are not
focusable renderables. Nothing restores it:
- autoFocus is false in the renderer (app.tsx:183)
- the prompt's createEffect only re-focuses when visible changes,
which sidebar clicks do not affect
- dialog.refocus() only fires on dialog close/escape, not sidebar clicks
- no sidebar onMouseDown handler called focus() on anything
Result: all keyboard input was silently discarded after any sidebar
click (toggle pack, expand/collapse section, dismiss getting-started).
The global useKeyboard handlers still fired but the textarea was
unfocused so typed characters never reached it, appearing as a total
freeze.
Fix: import usePromptRef in sidebar.tsx, add a refocusPrompt() helper
that calls promptRef.current?.focus(), and call it from every
onMouseDown handler in the component:
- MCP section header collapse toggle
- Knowledge Packs expand/collapse toggle
- Knowledge Pack row enable/disable toggle
- LSP section header collapse toggle
- Todo section header collapse toggle
- Modified Files section header collapse toggle
- Getting Started dismiss button
PromptRefProvider wraps the entire App tree (app.tsx:157) so
usePromptRef() is always in scope from the sidebar.
The previous commit applied refocusPrompt() to all sidebar onMouseDown handlers. Scope it back to only the two KP-specific handlers: the pack row enable/disable toggle and the expand/collapse button. These are the only new sections added as part of this feature. All other sidebar sections are unchanged from upstream.
…lapse compaction When a user types a message while the agent is still running, opencode re-parents subsequent assistant messages to the new user message. This caused detectChains to split what is logically one continuous work session into multiple separate chains, leading float sub-collapse to summarize them independently and lose cross-chain context. Fix: instead of breaking the chain walk on any user message, absorb mid-run user interjections (non-compaction-trigger user messages) into the current chain by tracking all chain user IDs in a Set. Assistant messages parented to any of those user IDs are still recognised as part of the same chain. A compaction trigger (parts.some p.type===compaction) still terminates the chain as before. Affects both float compaction (shouldFloatSubCollapse) and collapse compaction (blocking chain detection at extract boundary).
…ModelMessages Knowledge pack messages were loaded via KnowledgePack.fromSession() and stored in a sessionMessages variable that was never used. Both the plugin transform hook and toModelMessages received msgs without KP messages, causing KP content to be invisible to the LLM when plugins (like DCP) were active. Replace the dead sessionMessages clone with a direct unshift of KP messages into msgs so they flow through the plugin transform hook and into toModelMessages correctly.
Three changes to sidebar.tsx: 1. Remove double-refetch race in togglePack: the explicit refetchActive() call was redundant (kpMessageCount memo already triggers reactive refetch via sync store) and could destroy/recreate DOM elements while opentui's native layer was still processing the mouse event, corrupting the internal focus state machine and leaving currentFocusedRenderable permanently null. 2. Defer refocusPrompt with setTimeout(1) to match the dialog system's pattern -- opentui's native layer does post-callback processing (hover recheck, mouseUp dispatch) that can overwrite a synchronous focus(). Add a 50ms safety net to catch focus loss from async re-renders. 3. Add refocusPrompt() to all sidebar onMouseDown handlers (MCP, LSP, Todo, Diff, Getting Started) that were missing it, preventing focus loss when clicking any sidebar section.
…tate The previous fix removed refetchActive() entirely to prevent the DOM destruction race, but this meant the UI never updated the enabled/disabled visual state until the reactive kpMessageCount chain eventually propagated. Now togglePack chains .then(() => setTimeout(() => refetchActive(), 1)) so the refetch happens after both the server response AND outside opentui's mouse event processing window.
…emove When the user toggles a knowledge pack from the sidebar, opencode writes to the local project config. Since opencode does not merge the knowledge.packs array between global and project configs (project array fully overrides global), any globally-configured packs would be silently lost once the project file defines that key. Fix mirrors the approach used by --kp-add/--kp-remove in utils/coder: - ADD: reads global config packs and seeds them into the project file first, then appends the new pack. Globally-enabled packs that are not yet in the project file are mirrored with a log message. - REMOVE: reads only the project file and deletes the entry entirely. No global mirroring on remove (user only asked to remove one pack). Added Config.getProject() to read the local project config file without merging global config, required to inspect the current project state before seeding.
Config.update() was writing to Instance.directory/config.json but the
config loader only scans for opencode.jsonc and opencode.json — it never
reads config.json. So knowledge pack changes written via the sidebar were
silently dropped on the next session.
Write to {worktree}/.opencode/opencode.json instead, which is the path
the loader walks. Filesystem.write already creates parent dirs recursively
so .opencode/ is created automatically on first write.
Also fix Config.getProject() to read from the same corrected path.
If the project config has no packs array, or the pack being removed is not present in it, skip the write entirely. Avoids creating .opencode/opencode.json with an empty packs array when the user removes a globally-sourced pack that was never persisted locally.
Float sub-collapse was only reachable mid-chain (when finish=tool-calls) because the prompt loop exits early on stop finish before reaching the float pre-check at line 570. This meant complete chains accumulated untouched until context hit the collapse trigger threshold (0.87). Now the float pre-check runs inside the stop-exit branch before breaking, so sub-collapse fires on complete chains at every turn boundary. The loop still breaks after compacting -- we do not re-invoke the LLM -- leaving the context trimmed for when the next user message arrives.
…n count lastFinished was scanned backwards for the first assistant with any finish value (stop, tool-calls, end-turn, etc.). In long agentic sessions the last stop finish may be dozens of steps behind the current position -- e.g. 139k tokens at 69.7% when the session is actually at 174k (87%). Passing those stale tokens to floatModePreCheck caused the minFloat gate (0.7) to fail even though the session was well into the float window. Introduce lastWithTokens: the most recent assistant with non-zero total tokens (skipping aborted messages with total=0). Use lastWithTokens.tokens at all three token-sensitive sites: the stop-exit float pre-check, the mid-chain float pre-check, and the isOverflow check. lastFinished is kept for its existing use in the message-ordering filter at step > 1.
…low token count" This reverts commit d199a331089f1df635b11454043897fef3b57099.
…tant has finish=tool-calls Root cause of float compaction never firing (chainCount always 1): commit e28c27f70 introduced chainUserIds to handle genuine mid-run user interjections -- cases where the user types while the agent is still running tool calls, causing subsequent assistants to be re-parented to the new user message instead of the original chain anchor. The intent was correct: absorb those user messages into the current chain so the parentID walk continues through the re-parented assistants. However, the implementation had no guard on WHICH user messages qualify as interjections. It absorbed every user message encountered during the chain walk, regardless of whether the agent was actually mid-run or had already finished. This caused detectChains to treat sequential independent user turns (e.g. user sends a new task after agent stops) as interjections, merging all of them into one giant chain. Observed in session ses_33f374e7bffe (494 messages, post-compaction section rows 329-494): detectChains produced chainCount=1 spanning the entire session. With chainThreshold=2, shouldFloatSubCollapse returned null on every call. Float never fired despite 76% context usage. Confirmed from actual DB data: of the 11 user messages in rows 329-395, only ONE is a genuine mid-run interjection (row 393, preceded by a tool-calls assistant at row 392). All others are preceded by stop, end-turn, or summary assistants -- independent turns, not interjections. The fix: before absorbing a user message as an interjection, check that messages[j-1] is an assistant with finish==='tool-calls'. Any other predecessor (stop, end-turn, empty finish, summary) means the agent had already completed its turn, so this user message starts a new independent chain instead. With this fix, detectChains correctly breaks at each independent turn boundary, producing 7+ valid chains in the post-compaction section. chains.length > chainThreshold(2) becomes true, shouldFloatSubCollapse returns the oldest chain, and float sub-collapse fires as designed. This fix works in tandem with 2f2c86cc1 (stop-exit float pre-check): - This commit fixes chain counting so shouldFloatSubCollapse returns non-null (was the primary blocker) - 2f2c86cc1 ensures floatModePreCheck runs at stop/end-turn exits, not only during tool-calls continuations (secondary coverage gap)
bea7aa2dd added diagnostic log statements to detectChains to make the
float compaction decision process greppable from dev.log:
grep 'COLLAPSE detectChains' ~/.local/share/opencode/log/dev.log
Three tags were added:
COLLAPSE detectChains chain start
Logged when a new chain candidate begins (outer loop user message).
Fields: userIdx, userId.
COLLAPSE detectChains user boundary
Logged at every inner-loop user message encountered during a chain
walk. Shows whether it was absorbed as a mid-run interjection or
broke the chain.
Fields: userIdx, userId, prevRole, prevFinish, isInterjection.
isInterjection=true means prev assistant had finish=tool-calls and
the user message was absorbed into the current chain.
isInterjection=false means the chain walk stopped here.
COLLAPSE detectChains chain end
Logged after the inner walk finishes, before the chain is accepted
or discarded. Shows assistant count and whether the chain is valid
(>= 2 assistants required).
Fields: userIdx, userId, assistants, valid.
These logs were added to diagnose the float sub-collapse not firing.
The root fix is in 21f44b60b (interjection guard) and 2f2c86cc1
(stop-exit float pre-check). Compaction confirmed working: session
ses_33f374e7bffeJ9EzY5j3ZFDJ9y ran collapse at 87%, reduced from
174,350 to 134,429 tokens, then float correctly stayed idle below
minFloat=0.7 as context refilled.
…ation for collapse/float
Adds a dedicated compaction model selector so users can run one model
for chat and a different model for summarization (e.g. Claude Sonnet for
interactive coding, Zen Big Pickle free tier for zero-cost compaction).
Model resolution priority: TUI selection > agent.compaction.model config
> session model. When no TUI selection is set, behavior is identical to
upstream.
Changes:
- DialogModel gains target="compaction" prop — no duplicate component
- SessionCompaction.process() accepts optional compactionModel override
- CompactionPart schema extended with optional compactionModel field
- compaction_model_list keybind added (default: none)
- /compaction-models slash command and command menu entry
- local.model.compaction context backed by kv.signal('compaction_model')
- Prompt footer shows active compaction model when set
- SDK regenerated via ./script/generate.ts
ryangamerdev
force-pushed
the
feature/compaction-model
branch
from
August 26, 2026 01:20
8756834 to
4edb177
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes anomalyco#13946
Closes anomalyco#12135
Related: anomalyco#14368, anomalyco#14259, anomalyco#14233
Type of change
What does this PR do?
Adds a dedicated compaction model selector so users can run one model for chat and a different model for summarization. Run Claude Sonnet or Opus 4.6 for interactive coding while compacting with Zen Big Pickle (free) to eliminate compaction cost entirely, or GPT-5.3 Codex (400k context) for detail-oriented summaries that capture nuance your session model would lose — with headroom to compact sessions that have already exceeded their primary model's context limit.
For best results use this enhanced compaction prompt: anomalyco#14662
Why not just use
agent.compaction.modelin config?The existing
agent.compaction.modelconfig option is static — it requires editingopencode.jsonand is fixed for the lifetime of the config. This PR adds runtime switching without touching config files, matching how the primary model switcher works:Both approaches coexist. Model resolution priority: TUI selection > config file (
agent.compaction.model) > session model. When no TUI selection is made, the config-based approach works exactly as before. Nothing is broken.Features:
/compaction-modelsslash command and "Switch compaction model" in the command menukv.signalinkv.json, matching how all other TUI settings are persistedChanges:
DialogModelacceptstarget="compaction"prop — no duplicate componentSessionCompaction.process()accepts and resolves the override modelCompactionPartschema extended with optionalcompactionModelfieldcompaction_model_listkeybind added (default: none)./script/generate.tsHow did you verify your code works?
Tested with Claude Sonnet 4.5 (chat) + GPT-5.3 Codex (compaction). Confirmed correct model at each pipeline stage via
[compaction-model]log entries. Compaction completed successfully, session resumed on the original model. Also tested auto-compaction with no override set to verify identical upstream behavior. Verified that existingagent.compaction.modelconfig continues to work when no TUI override is set.Checklist