fix: rebuild oversized provider requests once without losing retained context - #52
Merged
Merged
Conversation
Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Adds a provider-budget preflight to the streaming loop. When an optional synchronous request_budget capability reports that a fully assembled request is oversized, the loop performs one request_retention budget rebuild, preserving the same human and reminder bodies and replaying request-only injections without rerunning hooks. If the required floor cannot fit, it raises ContextLengthError locally; it does not retry a real HTTP overflow. Providers without the capability, including Anthropic, preserve legacy behavior and byte parity.
Why
The companion producer change is merged: microsoft/amplifier-module-provider-openai#95. This loop-side guard consumes its optional capability so oversized assembled requests can be reduced once before SDK dispatch while retaining required context.
How to verify
DTU-native validation against Core 1.6.1 passed: Loop 295 tests including 21 focused; OpenAI 973 including 22 focused with 2 live tests deselected; Context 140 plus 1 expected failure; total 1408. Ruff passed for five changed Python files. Coverage includes paired real-context/StreamingOrchestrator/provider-adapter flows with fake SDKs only: baseline fake overflow versus one fitting candidate, required wire bodies exactly once, Anthropic byte parity, irreducible-floor SDK zero calls, and plausible usage calibration. The repository CI matrix runs Python 3.11 through 3.14 against amplifier-core 1.5.2.
Scope and limits
No Core, dependency, configuration-knob, or context-window changes. The guard can compact earlier on an initial or resumed turn; operational estimates are not a native 1M-context guarantee.
Breaking changes
None.