Skip to content

feat(models): support capabilities for custom models - #7124

Closed
Bil0000 wants to merge 18 commits into
pingdotgg:mainfrom
Bil0000:t3code/custom-model-capabilities
Closed

feat(models): support capabilities for custom models#7124
Bil0000 wants to merge 18 commits into
pingdotgg:mainfrom
Bil0000:t3code/custom-model-capabilities

Conversation

@Bil0000

@Bil0000 Bil0000 commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Problem

Custom model inference worked, but custom catalog entries had empty ModelCapabilities. The
composer therefore hid reasoning, speed, context-window, and other model controls. Claude 1M
context also required putting [1m] in the visible custom model ID.

Root cause

Provider settings stored custom model IDs only. Per-model descriptors never reached the provider
catalog, composer, ModelSelection, or adapter request path.

Design

Custom models can now store optional ModelCapabilities keyed by their exact model ID.

  • Reuses the existing ProviderOptionDescriptor and ModelSelection contracts.
  • Lets users add any current descriptor type: select or on/off.
  • Lets users set each descriptor's exact ID and label, add exact select values, choose defaults,
    and set boolean defaults.
  • Lets users copy one or more exact descriptors from any model catalog exposed by another supported
    harness instance.
  • Replaces an existing control with the same ID in place; newly copied IDs append.
  • Uses no model-name, provider, control-ID, or template allowlist in the editor or catalog.
  • Renders only the descriptors declared for that custom model.
  • Keeps arbitrary descriptors intact across settings, snapshots, and clients.
  • Keeps provider-specific request translation inside each adapter.
  • Preserves exact custom model IDs, including IDs that resemble built-in aliases.
  • Keeps built-in capability resolution unchanged and ahead of custom metadata.

An unknown descriptor can still be saved and shown. It has no request effect until that provider
adapter supports its ID. This keeps the UI generic without sending guessed provider options.

Provider behavior

Provider Descriptor IDs translated at runtime Translation
Claude effort, fastMode, contextWindow, thinking Claude SDK options and settings
Codex reasoningEffort, serviceTier Codex app-server options
Cursor reasoning, contextWindow, fastMode, thinking Matching ACP session configuration reported by the active Cursor CLI
OpenCode variant, agent OpenCode SDK request options
Grok None today Current Grok adapter sends model selection only

Claude fastMode is Claude Code's native setting. It is not OpenAI priority service. Codex speed
uses native serviceTier; T3 Code does not translate one into the other.

Cursor applies a selection only when the active CLI reports a matching ACP configuration option.
OpenCode has no speed-tier or context-window request option today. Grok has no model-option request
path today.

Copying preserves exact descriptor IDs and values. It does not translate meanings between
providers. The target adapter still decides whether a copied descriptor can affect its request.

Context-window handling

For Claude only, selecting a declared contextWindow=1m adds Claude Code's [1m] selector to the
request model ID. The stored and visible custom model ID stays exact. No model gets a global 1M
assumption.

Backward compatibility

  • Existing settings without capability metadata decode unchanged.
  • Existing custom models keep their prior behavior.
  • Existing select option labels stay intact; new values derive labels from their exact IDs.
  • New custom models start with an explicit empty capability set.
  • Explicit empty metadata stays distinct from absent legacy metadata.
  • Removing a custom model removes its saved capability metadata.
  • Existing built-in models and their capabilities stay unchanged.

UI

Custom-model rows now have a generic control editor. Select values render as tags. Space, comma,
Enter, or blur commits a value; clicking a tag selects its default; and its remove action deletes
it. Stored exact values, including values containing commas, remain intact.

Copy controls opens a harness picker, a model picker, and a checkbox list. Users can copy one or
more controls. Duplicate harness names include their instance ID. The model being edited is the
only excluded source. Catalog refreshes clear stale checked controls before a copy can occur.

Manual edits and copied controls stay local until Save is selected. Closing the editor without Save
cancels them. The boolean control heading is ON / OFF.

Web and desktop share this UI. Composer option rendering and selection storage continue to use the
existing shared descriptor system used by web, desktop, and mobile.

The author will attach an updated UI video separately.

Tests

Coverage includes:

  • old settings decoding and capability-map persistence
  • built-in precedence and exact custom capability resolution
  • reserved and inherited object-key safety
  • generic select/on-off descriptor creation, exact select-choice IDs, and composer visibility
  • tag entry delimiters, comma-bearing values, tag default selection, Save action, and ON / OFF
  • cross-harness source collection, exact multi-control copy, replacement order, self-source
    exclusion, duplicate harness labels, and catalog-refresh selection safety
  • explicit empty versus absent legacy metadata
  • exact alias-shaped custom IDs through start, model switches, and follow-up turns
  • Claude effort, fast-mode true/false, context, and thinking options
  • Codex reasoning and service-tier options
  • Cursor ACP and OpenCode SDK option paths
  • Grok's model-only limitation

Latest copy-controls verification:

  • vp test run apps/web/src/components/settings/ProviderModelsSection.test.ts apps/web/src/components/settings/ProviderInstanceCard.test.ts apps/web/src/components/settings/ProviderSettingsPanel.environment.test.tsx — 32 passed
  • vpr -F @t3tools/web typecheck — passed
  • focused vp lint ... --deny-warnings — passed
  • focused vp fmt ... --check — passed
  • git diff --check — passed

Earlier branch-wide verification before the UI-only follow-ups:

  • Focused capability suite — 355 passed across 15 files
  • vp check — 0 errors; 12 existing repository warnings
  • vpr typecheck — passed across 15 packages
  • vp run test — 8,215 passed; 7 skipped
  • vp run lint:mobile — passed; optional native linters are unavailable on this Linux host
  • vp run release:smoke — passed

Built with GPT-5.6-sol through the Codex harness.

Note

Add per-custom-model capability configuration across all providers

  • Introduces a CustomModelCapabilities schema (a map of model slug → ModelCapabilities) added to settings for all providers (Claude, Codex, Cursor, Grok, OpenCode).
  • Provider snapshot builders (providerModelsFromSettings) now assign declared per-model capabilities from this map instead of always using a shared fallback.
  • Claude adapter and text generation forward custom capabilities into model ID resolution (including [1m] suffix), effort normalization, context window selection, and fastMode/thinking boolean settings.
  • Codex session runtime preserves exact custom model IDs through normalization and propagates customModels to thread/turn start requests.
  • Settings UI gains a per-custom-model capabilities popover in ProviderModelsSection, with controls to add/edit select and boolean descriptors, copy from other providers, and reconcile capabilities when the custom model list changes.
  • applyServerSettingsPatch now replaces customModelCapabilities maps instead of deep-merging them, preventing stale keys from persisting across updates.
  • TraitsPicker visibility now responds to any declared descriptor, not just the hardcoded built-in controls.
  • Risk: customModelCapabilities patch semantics changed to replace-not-merge; existing tooling that relies on partial-patch behavior to update individual model capabilities will overwrite the entire map.

Macroscope summarized fd65aee.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fb85a802-814b-4720-9650-2cfa92eee7da

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Aug 15, 2026
Comment thread packages/contracts/src/settings.ts
Comment thread apps/web/src/components/settings/ProviderInstanceCard.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changed web UI (ProviderModelsSection.tsx, ProviderInstanceCard.tsx) against the shared primitive system. The new capability editor correctly reuses Popover, Checkbox, Switch, Select, DraftInput and Button primitives rather than reconstructing them, and Tailwind stays owner-local. Two consistency issues in the new custom-model controls popover are noted inline.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
@macroscopeapp

macroscopeapp Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature for configuring custom model capabilities, including new UI components (CustomModelCapabilitiesEditor, CustomModelCapabilityCopyPicker), new schema fields across 5 providers, and runtime behavior changes propagating configuration to provider adapters. New features with this scope warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Co-authored-by: Bilal Bakr <62337003+Bil0000@users.noreply.github.com>
Comment thread apps/server/src/provider/Layers/ClaudeProvider.ts Outdated
Comment thread apps/server/src/provider/Layers/ClaudeAdapter.ts Outdated
Comment thread apps/server/src/textGeneration/ClaudeTextGeneration.ts Outdated
Co-authored-by: Bilal Bakr <62337003+Bil0000@users.noreply.github.com>
Comment thread apps/web/src/components/settings/ProviderInstanceCard.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One finding on the new custom-model capabilities editor. The previously flagged DraftInput sizing and the missing gear tooltip are both resolved.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
t3-code Bot and others added 3 commits August 15, 2026 21:14
Co-authored-by: Bilal Bakr <62337003+Bil0000@users.noreply.github.com>
Co-authored-by: Bilal Bakr <62337003+Bil0000@users.noreply.github.com>
Co-authored-by: Bilal Bakr <62337003+Bil0000@users.noreply.github.com>

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two UI consistency issues in ProviderModelsSection.tsx that were fixed in later commits but re-introduced by the revert: remove unrelated custom model changes commit (head tree is back to the original state). Details inline.

Also still open from an earlier review: the Default SelectTrigger size="compact" inherits w-full min-w-36 (9rem) from the default variant while its grid track is 8rem, so the trigger overflows its column — worth neutralizing the min-width at the call site (e.g. className="min-w-0") as other size="compact" call sites do.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One finding: on/off controls authored with a free-form ID are dropped by the composer's visibility gate, so the new editor can declare a control the composer never shows.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One finding in the new custom-model capability editor: edits reorder the declared controls, which moves the edited card to the bottom of the popover and changes control order in the composer.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx
@github-actions github-actions Bot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Aug 16, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changed web UI (ProviderModelsSection.tsx, ProviderInstanceCard.tsx, TraitsPicker.tsx, providerModels.ts) against the shared primitive contracts. One finding: the new tag-style "Values" field re-creates the Input/ComboboxChips container styling but drops its focus treatment, so the inline text input has no visible focus indicator.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx Outdated
Comment thread apps/web/src/components/settings/ProviderModelsSection.tsx
@juliusmarminge

Copy link
Copy Markdown
Member

Superseded by merged #807 (custom model names and option descriptors).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants