fix(api): reject doubled provider prefixes in agent model ids (#334)

[w7-mgfcode]

Summary

Closes #334 — Reliability E1 (Foundation epic of umbrella #380). Implements PRPs/PRP-reliability-E1-doubled-provider-prefixes.md.

A model identifier whose model-name part starts with another known provider prefix (e.g. google-gla:google-gla:gemini-3-flash-preview) previously passed validation, persisted via PATCH /config/ai, and only failed far downstream as a Gemini 404 at agent-run time. pydantic-ai 1.96.0 parse_model_id passes everything after the first colon verbatim, so the validation burden is entirely on this repo's config layer.

Changes

• app/core/config.py — validate_model_identifier now rejects a nested provider prefix inside the model name with a "Did you mean <corrected>?" message. Ollama name:tag identifiers (ollama:llama3.1:8b, ollama:gemma4:e2b) stay valid — the rule fires only when the first segment of the model name is itself a known provider. Unknown-provider check still outranks shape (pinecone:pinecone:x → "Unknown provider").
• app/features/config/service.py — apply_overrides_on_startup re-validates the two model-id override keys; a malformed persisted row is skipped with config.override_invalid_model_id (env/default stays in effect; the no-crash-on-startup contract stands).
• Tests — extended at all three layers the issue demands: validator unit tests (5 doubled-prefix rejections, mixed-prefix, suggestion message, 3 ollama-tag acceptances, ollama:openai edge), strict-mode model_validate JSON-path cases, Settings-boot rejection + acceptance, route-level 422 problem+json asserting code/errors[0].field/message, and a startup skip-and-warn service test with settings-singleton restore.

Validation

• ruff check + ruff format --check clean; mypy --strict + pyright --strict clean (0 errors)
• pytest -m "not integration": 1888 passed (pre-existing ollama:qwen3:8b consumer fixtures untouched and green)
• Config-slice integration tests vs real Postgres: 3 passed
• Live curl matrix on a fresh uvicorn: doubled prefix → 422 problem+json with "Nested provider prefix … Did you mean 'google-gla:gemini-3-flash-preview'?"; anthropic:claude-sonnet-4-5 → 200; ollama:llama3.1:8b → 200

Out of scope (per PRP)

Frontend input sanitization in ai-models-panel.tsx; #335 fallback-failure surfacing (E2); normalization mode; backfill of existing malformed app_config rows.

Summary by Sourcery

Reject invalid agent model identifiers with nested provider prefixes and guard persisted overrides from applying malformed IDs at startup.

Bug Fixes:

• Prevent agent model identifiers whose model-name segment starts with a known provider prefix from passing validation, while keeping valid multi-colon Ollama tag identifiers accepted.
• Ensure doubled-prefix model IDs are rejected consistently at schema, settings, and HTTP route levels, returning structured 422 validation errors instead of failing at provider runtime.
• Skip and warn on invalid persisted model-id overrides during startup so malformed rows never override environment or default configuration.

Enhancements:

• Document and codify the reliability change in a new PRP describing the doubled-provider-prefix hardening and its validation surface.
• Clarify validation error messaging by suggesting the likely correct model identifier when a nested provider prefix is detected.

Tests:

• Extend validator, schema, settings, service, and route tests to cover doubled and mixed provider prefixes, Ollama tag carve-outs, startup override skip behavior, and the 422 problem+json error shape for invalid model IDs.

[sourcery-ai]

Reviewer's Guide

Adds config-layer validation to reject agent model identifiers whose model-name segment starts with a second known provider prefix, and ensures invalid persisted model-id overrides are skipped on startup, with comprehensive tests across validator, schema, settings, route, and service layers, plus documentation via a PRP file.

Sequence diagram for apply_overrides_on_startup re-validating model-id overrides

sequenceDiagram
    participant ConfigService
    participant Validator
    participant Settings
    participant Logger

    ConfigService->>ConfigService: apply_overrides_on_startup(db)
    ConfigService->>ConfigService: iterate overrides.items()

    alt key in _MODEL_ID_KEYS and isinstance(value, str)
        ConfigService->>Validator: validate_model_identifier(value)
        alt validate_model_identifier raises ValueError
            ConfigService->>Logger: warning(config.override_invalid_model_id)
            ConfigService->>ConfigService: continue loop (skip override)
        else validate_model_identifier returns
            ConfigService->>Settings: setattr(settings, key, value)
        end
    else other override keys
        ConfigService->>Settings: setattr(settings, key, value)
    end

Loading

Flow diagram for validate_model_identifier nested provider prefix rejection

flowchart TD
    Start([Input model identifier v])
    CheckColon{"v contains ':'?"}
    Start --> CheckColon
    CheckColon -- No --> ErrNoColon["raise ValueError: expected 'provider:model-name'"]
    CheckColon -- Yes --> Split["provider, model_name = v.split(':', 1)"]

    BlankModel{"model_name blank?"}
    Split --> BlankModel
    BlankModel -- Yes --> ErrBlank["raise ValueError: blank model name"]
    BlankModel -- No --> ProviderAllowed{"provider in VALID_MODEL_PROVIDERS?"}

    ProviderAllowed -- No --> ErrUnknown["raise ValueError: Unknown provider"]
    ProviderAllowed -- Yes --> NestedColon{": in model_name?"}

    NestedColon -- No --> Ok["return v (valid identifier)"]
    NestedColon -- Yes --> GetNested["nested = model_name.split(':', 1)[0]"]

    IsProviderNested{"nested in VALID_MODEL_PROVIDERS?"}
    GetNested --> IsProviderNested

    IsProviderNested -- No --> Ok
    IsProviderNested -- Yes --> Suggest["suggested = provider + ':' + model_name.split(':', 1)[1]"]

    Suggest --> ErrNested["raise ValueError: Nested provider prefix + 'Did you mean ' + suggested"]

    Ok --> End([Return unchanged v])
    ErrNoColon --> End
    ErrBlank --> End
    ErrUnknown --> End
    ErrNested --> End

Loading

File-Level Changes

Change	Details	Files
Harden model identifier validation to reject nested provider prefixes while preserving valid multi-colon Ollama tags.	Extend validate_model_identifier to detect when the model-name contains a nested provider prefix (e.g. 'google-gla:google-gla:…') and raise a ValueError with a suggested corrected id. Ensure the unknown-provider check continues to take precedence over shape errors (e.g. 'pinecone:pinecone:x' still reports unknown provider). Update the validate_model_identifier docstring to document the new nested-prefix rule and the Ollama tag carve-out.	app/core/config.py
Guard startup override loading so malformed model-id overrides never become active settings.	Introduce a _MODEL_ID_KEYS frozenset listing the model-id-related override keys to guard. In apply_overrides_on_startup, re-validate string overrides for these keys using validate_model_identifier and skip-and-warn on ValueError, preserving env/default values and never raising. Log a structured warning with code 'config.override_invalid_model_id' when an invalid override is skipped.	app/features/config/service.py
Expand tests to cover nested-prefix rejection, Ollama tag acceptance, schema/route/settings behavior, and startup override skipping.	Add validator-focused tests covering doubled and mixed provider prefixes, the correction hint message, acceptance of multi-colon Ollama identifiers, and models literally named like providers. Add AIModelConfigUpdate model_validate tests ensuring both agent_default_model and agent_fallback_model reject doubled prefixes via the JSON/validate_python path. Add Settings-level tests verifying doubled prefixes fail at settings boot while Ollama tag identifiers succeed when _env_file=None is used. Add route tests asserting PATCH /config/ai responds with a 422 problem+json on doubled prefixes, including the correct code, field, and error message contents. Add a service test for apply_overrides_on_startup that mocks persisted overrides, asserts invalid model-id overrides are skipped while other keys apply, and verifies a single warning log; ensure the Settings singleton is restored after mutation.	app/features/config/tests/test_schemas.py app/features/config/tests/test_service.py app/features/agents/tests/test_config_validation.py app/features/config/tests/test_routes.py
Document the reliability work and constraints for rejecting doubled provider prefixes in a PRP design document.	Add a PRP markdown file describing the goal, behavior changes, design decisions (reject vs normalize), startup guard semantics, test and validation plans, and out-of-scope items for this reliability epic. Capture relevant context such as library behavior, existing consumers, known gotchas, and a detailed implementation and validation blueprint.	PRPs/PRP-reliability-E1-doubled-provider-prefixes.md

Possibly linked issues

• fix(config): reject doubled provider prefixes in agent model ids #334: PR implements stricter validate_model_identifier logic and startup guards to reject doubled provider prefixes as described.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: da42c698-a927-40f4-930f-7dc1cdefbcf9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/config-reject-doubled-provider-prefix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}