Amplifier Simple Context Manager Module

Basic message list context manager for conversation state.

Prerequisites

• Python 3.11+
• UV - Fast Python package manager

Installing UV

# macOS/Linux/WSL
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

Purpose

Provides straightforward in-memory conversation context management. This is the reference implementation and default context manager.

Contract

Module Type: Context Mount Point: contexts Entry Point: amplifier_module_context_simple:mount

Behavior

• In-memory message list
• No persistence across sessions
• Automatic compaction when approaching token limit (keeps system messages + last 10 messages)
• Preserves tool pairs as atomic units during compaction (data integrity guarantee)
• Optional real-usage token meter (token_meter: "actual", default off) drives the compaction trigger from real provider usage instead of the built-in estimator -- see Real-usage token meter below

Configuration

[[contexts]]
module = "context-simple"
name = "simple"

[contexts.config]
max_tool_result_bytes = 131072  # Optional override; default is 128 KiB

Tool-result text ingress cap

Before admission, a tool message's string content (including JSON serialized ToolResult dict/list output) is limited to max_tool_result_bytes, which defaults to 131,072 UTF-8 bytes. Oversized content keeps a UTF-8-safe prefix and one explicit retrieval marker. Raise this one explicit setting only for a legitimate larger text result; there is no off switch in this version.

For block content, the cap covers only direct text blocks. Image, audio, unknown, and nested block data are not read or sliced. The original oversized text is not retained in metadata, a side file, or the admitted transcript: retrieve missing content using narrower read/query parameters; do not repeat state-changing actions just to recover output. For ill-formed Python text containing lone surrogates, byte accounting uses UTF-8 replacement; under-cap content remains unchanged, while an oversized clipped prefix is valid UTF-8.

The default is a finite observed baseline, not a universal tail guarantee: 509 outputs from 16 stock-main S1 captures had a 40,139-byte p99 and 87,301-byte maximum, with none above 128 KiB. 128 KiB is 3.27x that p99 and 1.5x that maximum, while still fitting the local 64k-window regression where a 256 KiB cap would not.

Usage

# In amplifier configuration
[session]
context = "context-simple"

Perfect for:

• Development and testing
• Short conversations
• Stateless applications

Not suitable for:

• Cross-session persistence
• Custom compaction strategies

Compaction Strategy

The SimpleContextManager uses ephemeral compaction - get_messages_for_request() returns a compacted VIEW without modifying the admitted internal message history. Ingress-clipped tool text is irreversible and is not retained in the canonical transcript; compaction remains view-only.

Compaction triggers when token usage reaches the configured threshold (default: 92% of the effective budget -- which is derived from the provider, not from max_tokens; see Where the compaction trigger comes from):

Protected Messages (Never Removed)

• System messages: All system messages are always preserved
• First human prompt: The original human task/request is protected, using message metadata rather than treating every user-role message as human input
• Last human prompt: The most recent human input is protected by the same metadata-based classification
• Recent messages: Last N% of messages (configurable via protected_recent)
• Recent tool results: The last protected_tool_results results (default 5) are protected from both truncation and removal; a protected sibling also prevents removal of its owning call group
• Tool pairs: Tool_use and tool_result messages are treated as atomic units

Request-scoped retention

The optional context.request_retention capability lets an orchestrator name the exact persisted reminder bodies required for its next request. The newest matching admitted ephemeral=True, persisted=True user-role envelope is kept complete through compaction, along with the first/latest human prompts. Quoting reminder XML in an ordinary human prompt does not change its identity.

This protects delivery in the request view; it does not change message roles, pin every historical reminder, or rewrite canonical history. A missing required body or an irreducible required set that cannot fit raises ContextLengthError instead of silently dropping instructions. Failed assembly before compaction event delivery restores the prior compaction state.

Protection takes precedence over the compaction target. A protected tool cohort can leave a view above that target; it is not a strict native-token ceiling.

After a compaction decision, every later provider-facing request reuses the same reduced view before it is measured; canonical history remains complete. The capability also accepts hard_fit=True for a provider-directed forced rebuild: it targets the supplied effective request budget rather than applying target_usage again. This is an additive capability keyword for orchestrators, not a user configuration setting; ordinary token_budget calls keep their existing target_usage semantics.

For a hard-fit request, sticky compaction decisions and their accounting roll back if assembly fails or is cancelled before the final, notice-inclusive view starts context:compaction event delivery. Once delivery starts, that validated compaction is retained even if the caller is cancelled. The event marks this compaction commit boundary only; it does not imply that a provider request was dispatched.

Compaction Phases

1. Phase 1 - Tool Result Truncation: Older tool results are truncated to reduce token usage
2. Phase 2 - Message Removal: Older non-protected messages are removed if still over budget

Tool Pair Preservation

Anthropic API requires that every tool_use in message N has a matching tool_result in message N+1. The context manager preserves these pairs as atomic units during compaction to maintain conversation state integrity and prevent API errors.

Critical implementation detail: When an assistant message has multiple tool_calls, there are multiple consecutive tool_result messages after it. The compaction logic walks backwards through these tool results to find the originating assistant message, ensuring the entire tool group is preserved as an atomic unit. This prevents orphaned tool results that would cause API validation errors.

Where the compaction trigger comes from

The trigger is one multiplication:

trigger = compact_threshold * effective_budget

effective_budget comes from _calculate_budget(), in this priority order:

1. an explicit token_budget= argument (deprecated, rarely used);
2. provider.get_model_info() -> context_window - 0.5 * max_output_tokens - 4096;
3. provider.get_info().defaults -> the same formula;
4. only if none of the above yields a window: the configured max_tokens.

max_tokens is a fallback, not a cap

Orchestrators call get_messages_for_request(provider=provider), so in practice branch 2 or 3 always answers and branch 4 is never reached. The max_tokens value in your bundle config (the shipped foundation bundle sets max_tokens: 300000) therefore has no effect on when compaction fires. Lowering it to compact sooner, or raising it to compact later, is a no-op on the wire.

This is a real trap, not a theoretical one. The cadence probe that produced the numbers below could not move the trigger with config at all: its harness had to patch this module's source in-container to add budget = min(budget, self.max_tokens) before either of its arms would compact in a bounded run.

tests/test_compaction_trigger_provenance.py pins this behavior in both directions -- same history and same config compacts with no provider and does not compact with one -- so the trap fails a test rather than a measurement run.

To move the trigger, move compact_threshold. It is the only shipped knob that expresses "compact later" independently of the provider, and the old value stays reachable:

context:
  module: context-simple
  config:
    compact_threshold: 0.80   # compact earlier than the 0.92 default

What the cadence measurement does and does not say

Measured on the S5-CRAC scenario (gpt-5.6-terra, n=2 vs n=5 reused baselines; capture root .amplifier/evaluation/treatment-validation/20260901-cadence/, PROBE4-VERDICT.md), raising the compaction trigger budget from 45,000 to 70,000 tokens produced:

arm	boundaries	requests	wall (s)	cost ($)	S5 score
cad-today (trigger 45k, n=5)	21.6	104	562	2.58	94.4
cad-fewer (trigger 70k, n=2)	9.5	74	485	2.65	95.0

-29% requests, -14% wall, at equal cost and equal quality -- and the only arm in that matrix where input-item caching measurably occurred (20/72 and 11/77 requests). Buy the latency and request-count win; do not promise a cost win ($2.65 vs $2.58 is nil, in the wrong direction).

Two limits on that result, both from its own source:

• Both values are scenario forcing knobs. 45,000 exists to make a bounded 10-turn run compact at all. Neither is a production default, and neither is a value this module has ever shipped.
• Production already compacts later than cad-fewer did. With a 200,000-token window the shipped trigger is 0.92 * 163,904 = 150,791 tokens; cad-fewer's was 0.92 * 70,000 = 64,400. Adopting 70,000 as a budget cap would move the trigger earlier for every provider whose window exceeds it -- more boundaries, inverting the measured win. The parametrized test at the bottom of tests/test_compaction_trigger_provenance.py asserts exactly this. (Those trigger figures are compact_threshold * budget; get_messages_for_request also subtracts the 800-token compaction-notice reserve first, moving each down by 736 tokens. No ratio changes.)

The general finding still holds and is the one to carry forward: fewer compaction boundaries buys latency and request count, not money, and costs no measurable quality (post-compaction retention was 20/20 in every run of every arm, b_constraints 40/40 throughout -- on a scenario whose 5 crisp constraints may be a ceiling effect).

Real-usage token meter (token_meter)

The problem

The compaction trigger described above runs entirely off _estimate_tokens() -- len(str(msg)) // 4 over the Python repr() of each message. This estimator is never reconciled against what the provider actually billed anywhere in this module. In production sessions it has been measured roughly 2x off from real provider usage. Because the trigger and the whole progressive-compaction sizing logic are built on this number, running compaction any closer to the real ceiling than the current conservative default (92%) is unsafe on an estimator that inaccurate -- you would risk provider-side context-length rejections with no warning.

A companion module, amplifier-module-context-handoff, solved this for its own (non-compacting) reserve trigger by registering a listener on the canonical llm:response event and reading the provider's own reported usage instead of guessing. This module ports that same _on_llm_response meter, adapted to context-simple's compaction trigger.

What it does

• When hooks are available, this module always registers a listener on llm:response and records the provider's own reported usage for the most recent request: input_tokens + cache_write_tokens. Per the provider contract, input_tokens is the GROSS total (fresh + cache_read combined) billed as input; cache_write_tokens is billed disjointly (a first-time cache write of a large system/tool prompt can be billed almost entirely as cache_write_tokens with input_tokens near zero), so it must be added separately or true context-window occupancy would be undercounted by orders of magnitude. cache_read_tokens is not added again -- it is already inside the gross input_tokens figure.
• This recording happens regardless of token_meter mode -- it is a cheap, side-effect-free observability signal, exposed via context._last_token_meter_stats (populated on every get_messages_for_request() call, not only when compaction fires) so the estimator-vs-real drift is visible even in the default mode.
• Set token_meter: "actual" in config to additionally have the compaction trigger -- and _compact_ephemeral's internal escalation gate -- use that real measurement once at least one llm:response has been observed this session. Before the first response (or whenever hooks/events are unavailable), "actual" mode falls back to the same estimator "estimate" mode always uses.
• Default is token_meter: "estimate", which is byte-identical to this module's behavior before this meter existed -- verified by running the full pre-existing test suite unchanged. An unrecognized token_meter value logs a warning and falls back to "estimate" rather than raising.

Known, accepted limitation

Only the escalation gate (whether to compact at all, and whether a sticky escalation needs to advance) uses the real measurement in "actual" mode. The amount of reduction -- target_tokens and every per-level termination check inside _compact_ephemeral -- is still computed from the estimator throughout, because a real, provider-billed token count for a hypothetical smaller message set does not exist without another round trip to the provider. If the real measurement and the estimator disagree sharply, "actual" mode can still converge at level 1 without having done much real reduction (the estimator's own view already looked small enough). This module fires the escalation honestly in that case, but the sizing of that escalation is only as good as the estimator was before this meter existed. This mirrors context-handoff's own documented limitation that its measurement is retrospective (one-call lag): the meter describes the request that was just answered, not the one currently being assembled.

Future default flip (pending validation)

token_meter defaults to "estimate" in this PR specifically so it ships with zero behavior change. Flipping the default to "actual" -- and potentially raising compact_threshold closer to the real ceiling now that it can be measured accurately -- is a follow-up, not part of this change. It should happen only after running the module's own eval harness against "actual" mode's stats (_last_token_meter_stats) to confirm the expected reduction in compaction cadence (request count / wall time) holds up without a corresponding quality regression.

Dependencies

• Host-provided amplifier-core>=1.6.1, including amplifier_core.llm_errors.ContextLengthError for fail-loud request retention. Core remains provided by the host, not installed as a runtime module dependency.

Development and CI pin the released amplifier-core==1.6.1 package in uv.lock. A Git main source mapping can retain an older commit in the lockfile; the release pin ensures tests exercise the Core API required by this module.

uv sync --locked --all-extras --dev
uv run --locked pytest -q

Contributing

Note

This project is not currently accepting external contributions, but we're actively working toward opening this up. We value community input and look forward to collaborating in the future. For now, feel free to fork and experiment!

Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Contributor License Agreements.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.