Skip to content

fix(opencode): apply long context config pricing - #42919

Closed
dpclark4 wants to merge 2 commits into
anomalyco:devfrom
dpclark4:fix-200k-pricing
Closed

fix(opencode): apply long context config pricing#42919
dpclark4 wants to merge 2 commits into
anomalyco:devfrom
dpclark4:fix-200k-pricing

Conversation

@dpclark4

@dpclark4 dpclark4 commented Aug 16, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #42910

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

This fixes a bug where locally computed costs do not account for context tiered pricing. For models with higher pricing above a context threshold (e.g. 200k), the local pricing logic continued using the lower tier after the session crossed that threshold.

If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!

How did you verify your code works?

Wrote two tests to verify locally with tiered Grok model:

  • 199K and 200K tokens use the base rate. 201K tokens uses the long context rate
  • existing long context pricing remains when config does not provide an override

If we want extra confidence, I can build an opencode version off of this change, and retest the grok scenario I flagged in the linked issue. Let me know.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

If you do not follow this template your PR will be automatically rejected.

@dpclark4 dpclark4 changed the title fix(opencode): apply long-context config pricing fix(opencode): apply long context config pricing Aug 16, 2026
@github-actions github-actions Bot added needs:compliance This means the issue will auto-close after 2 hours. and removed needs:compliance This means the issue will auto-close after 2 hours. labels Aug 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference; please use your judgment.

  1. packages/opencode/src/provider/provider.ts:1450 — partial config clobbers the whole long-context block: context_over_200k: { input: 4, output: 12 } (no cache fields) turns cache_write into 0 even when the catalog supplied non-zero >200k cache prices via experimentalOver200K — consider per-field merge ({ ...existingModel?.cost.experimentalOver200K, ...mappedLongContext }) so users can override just input/output, matching how the base cost fields fall back per-key below.

  2. packages/opencode/test/provider/provider.test.ts:999 — the math pins that exceeding 200k re-prices the entire input (201_000 × 4/M = 0.804, not marginal) — that's correct for Grok/xAI but the generic experimentalOver200K name is also consumed by other providers where >200k pricing is typically marginal (Anthropic); worth one sentence in the config docs stating this is a whole-request switch so nobody reuses the field expecting tiered math.

  3. Nit — packages/opencode/test/provider/provider.test.ts:987 — estimate(200_000) sitting exactly at the boundary is great; add the mirror case one token below with output tokens to lock that output-side pricing also switches only past the threshold.

Overall: clean fix connecting an advertised-but-unwired config knob, with tests that assert both the wiring and the resulting billing math including the boundary. Item 1 is the only behavioral concern. Thanks!

@xyzs996

xyzs996 commented Aug 22, 2026

Copy link
Copy Markdown

The boundary case in the new test is pinned to the wrong side for xAI specifically, and it's worth catching now because the test will freeze it.

expect(estimate(200_000)).toBe(0.4)   // 200,000 × $2/M — base rate

xAI documents the opposite:

requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request

https://docs.x.ai/docs/models (read 2026-08-22; the note sits under the text pricing table, and the tables give 200k prompt tokens as the cutoff — e.g. grok-4.6 going $2.00 → $4.00 per million input). "Reaches" is >=, so a prompt of exactly 200,000 is billed at the long-context rate. That line should be toBe(0.8).

The cause isn't in this PR — session.ts:384 already reads contextTokens > 200_000, and the generic ladder above it reads contextTokens > item.tier.size. This PR just wires the config through and adds the first test that lands exactly on the line, so it's the point where the off-by-one gets locked in.

The fix is not a global >>=. Vendors genuinely disagree about which side the threshold token falls on, so the comparison can't be a constant in the ladder code:

vendor documented wording prompt exactly at threshold
xAI "requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request" long-context
Google (Gemini 2.5 Pro) $1.25, prompts <= 200k tokens / $2.50, prompts > 200k tokens standard
OpenAI (gpt-5.4/5.5) "prompts with >272K input tokens are priced at 2x input and 1.5x output for the full session" standard
MiniMax (M3) ≤ 512k input tokens / > 512k input tokens standard

All read 2026-08-22: https://docs.x.ai/docs/models, https://ai.google.dev/gemini-api/docs/pricing, https://developers.openai.com/api/docs/models/gpt-5.4, https://platform.minimax.io/docs/guides/pricing-paygo.

So contextTokens > item.tier.size is correct for three of the four and wrong for the one this PR is actually about. The narrow fix is >= for the experimentalOver200K path only (today that field is xAI-shaped — the name literally encodes xAI's 200k); the durable fix is carrying the side on the tier, e.g. tier: { type: "context", size: 200_000, inclusive: true }, so each catalog row maps onto a sentence someone can re-check instead of onto a 199_999 that a later cleanup will round back up.

This cuts both ways, which is why it's worth encoding rather than picking a house style — the same one-token error exists in the other direction elsewhere: Helicone's getPricingTier uses value >= threshold for the 200k Google models (Helicone/helicone#5788) and unsloth's studio pricing uses >= lc_thresh for the 272k OpenAI models (unslothai/unsloth#9527) — both start the expensive tier one token early, because they assumed the same convention globally in the opposite direction.

Re the review's item 3 above — a mirror case one token below is worth adding, but note 199_999 and 200_000 should now give different answers for xAI, so it's estimate(199_999) === 0.399998 / estimate(200_000) === 0.8, not a pair that both sit on the base rate.

Also worth flagging alongside item 2: for xAI the whole-request switch is right, and it's right for OpenAI too — "2x input and 1.5x output for the full session", with the input length selecting the output rate. That's what costInfo.output already does here. It is not right to assume marginal/graduated math for any of the four above; all four say the whole request re-prices.

I keep the cross-vendor version of that table with each row carrying the sentence it came from, if it's useful while deciding the tier shape: https://xyzs996.github.io/llm-api-pricing/prices.html#same-number-opposite-answer. Happy to send the boundary test cases as a follow-up PR if you'd rather keep this one scoped to the config wiring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect session cost estimate for models with context tiered pricing

3 participants