feat: model hub visibility & features + review fixes

[sheepdestroyer]

This PR contains the complete implementation for Model Hub visibility and features, incorporating all fixes and feedback from the review of PR #14.

Summary by Sourcery

Enhance model visibility and capabilities metadata across LiteLLM's Model Hub and routing stack, including OpenRouter and Ollama models, while aligning docs, verification scripts, and CI dependencies with the new behavior.

New Features:

• Expose public model groups and capabilities metadata for key OpenRouter, Ollama, and agent model groups so they appear correctly in the LiteLLM Model Hub.
• Register static Ollama DeepSeek deployments in the LiteLLM database at startup to surface accurate features, token limits, and pricing in the model hub.
• Propagate model capabilities (vision, reasoning, tools, context length) when auto-registering free OpenRouter models into LiteLLM deployments.

Enhancements:

• Refine synthetic llm-routing-* model context lengths to match their downstream targets.
• Factor out shared database cleanup logic for purging stale LiteLLM deployments before re-registration.
• Modernize verification scripts to use httpx for metrics and completion requests with clearer error reporting.

CI:

• Install httpx in the test workflow so verification scripts run successfully in CI.

Documentation:

• Document updated context lengths and add a note pointing users to LiteLLM's Model Hub table UI for inspecting model capabilities.

[sourcery-ai]

Reviewer's Guide

Implements full Model Hub visibility and feature metadata for routing and agent models, including DB-backed registration for Ollama deployments, explicit model_info for key LiteLLM model groups, updated context lengths/documentation, and modernized verification scripts using httpx, plus CI wiring for their dependencies.

Sequence diagram for Ollama model DB registration during router startup

sequenceDiagram
    participant FastAPIApp
    participant lifespan
    participant _register_ollama_models_in_db
    participant _purge_stale_deployments
    participant Postgres
    participant LiteLLMGateway

    FastAPIApp->>lifespan: start
    lifespan->>_register_ollama_models_in_db: await _register_ollama_models_in_db(litellm_master_key)

    _register_ollama_models_in_db->>_purge_stale_deployments: _purge_stale_deployments(db_url, ollama-deepseek-%)
    _purge_stale_deployments->>Postgres: DELETE FROM LiteLLM_ProxyModelTable WHERE model_name LIKE ollama-deepseek-%
    Postgres-->>_purge_stale_deployments: rows deleted

    loop for payload in ollama_models
        _register_ollama_models_in_db->>LiteLLMGateway: POST /model/new (payload with model_info)
        LiteLLMGateway-->>_register_ollama_models_in_db: 200/201 Created
    end

    _register_ollama_models_in_db-->>lifespan: registration summary
    lifespan-->>FastAPIApp: startup complete

Loading

File-Level Changes

Change	Details	Files
Refactor roster sync to persist rich model capabilities into LiteLLM deployments and centralize DB cleanup.	Introduce _purge_stale_deployments helper using asyncpg to delete deployments by pattern from LiteLLM_ProxyModelTable. Extend sync_adaptive_router_roster to collect per-model context lengths and supported params from OpenRouter metadata. Include model_info (vision, reasoning, tools support, mode, token limits, public flag) when registering tiered agent-* deployments via /model/new. Reuse the new purge helper to clean agent-* deployments before roster sync instead of duplicating asyncpg logic.	router/main.py
Register Ollama DeepSeek models as first-class DB models with explicit capabilities and pricing, and wire registration into app startup.	Add _register_ollama_models_in_db which defines static ollama-deepseek-* payloads with litellm_params and detailed model_info, purges stale DB entries, and calls /model/new to upsert them. Invoke Ollama DB registration during FastAPI lifespan startup after roster sync, with non-fatal error handling and logging.	router/main.py
Expose model visibility and capability metadata to LiteLLM via config, including public model groups and model_info for key routing and agent entries.	Add litellm_settings.public_model_groups listing openrouter-auto, llm-routing-ollama, ollama-deepseek-* and agent-* cores. Attach model_info blocks to openrouter-auto, llm-routing-ollama, ollama-deepseek-v4-pro/flash, and each agent-* core, defining capabilities, token limits, pricing (where applicable), and public flags. Align context limits in config with downstream providers, e.g., 262K for OpenRouter auto and 512K for Ollama DeepSeek models.	litellm/config.yaml
Adjust routing model metadata and documentation to reflect accurate context lengths for different routing tiers.	Update injected routing model stubs in proxy_models() to set differentiated context_length values for auto-free/agy, Ollama-based, and agy-only routing models. Synchronize the README Model Catalog table to show the new context limits for routing and agent-core models. Add a README tip pointing users to LiteLLM's Model Hub Table UI for inspecting capabilities, limits, and costs.	router/main.py README.md
Modernize verification scripts to use httpx, improve error reporting, and ensure CI has dependencies.	Replace urllib.request usage in verification scripts with httpx for both metrics (GET) and gateway calls (POST), including timeout handling and JSON parsing. Improve error handling by raising on HTTP errors, printing response bodies for debugging, and simplifying exception paths. Update the GitHub Actions test workflow to install httpx so verification scripts run in CI.	scripts/verification/verify_ollama_cooldown.py scripts/verification/verify_direct_ollama_cooldown.py scripts/verification/verify_ollama_routing.py .github/workflows/test.yml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cd46e236-3153-462c-920d-1f5dccb39882

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/model-hub-visibility-review-fixes

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request updates model context lengths, adds detailed model capabilities and token limits to LiteLLM configurations, and introduces automatic database registration for static Ollama models. Additionally, verification scripts are refactored to use httpx instead of urllib.request. The review feedback identifies a critical NameError in router/main.py where supported_params is referenced instead of supported_parameters, which will crash the application on startup. It also recommends dynamically loading Ollama model configurations from config.yaml to avoid code duplication and adhere to DRY principles.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Critical Bug: NameError on Startup

The variable supported_params is not defined in this scope. The actual variable defined at line 393 is supported_parameters.

Using supported_params here will raise a NameError at runtime during the roster sync on startup, crashing the application. Please update it to reference supported_parameters instead.

Suggested change

	model_contexts[mid] = m.get("context_length") or 262144
	model_supported_params[mid] = supported_params
	model_contexts[mid] = m.get("context_length") or 262144
	model_supported_params[mid] = supported_parameters

[gemini-code-assist]

Maintainability: Avoid Configuration Duplication (DRY)

The static configurations for ollama-deepseek-v4-pro and ollama-deepseek-v4-flash are completely duplicated here in Python code, matching the definitions in litellm/config.yaml exactly. This is highly error-prone and violates the DRY (Don't Repeat Yourself) principle.

Since the LiteLLM configuration is already loaded into the global config variable at startup, you can dynamically extract these models using a list comprehension. This ensures that any future updates to model parameters, costs, or timeouts in config.yaml are automatically reflected in the database registration without requiring duplicate updates.

    ollama_models = [
        m for m in config.get("model_list", [])
        if isinstance(m, dict) and m.get("model_name", "").startswith("ollama-deepseek-")
    ]

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The new _register_ollama_models_in_db helper hardcodes admin_url (127.0.0.1:4000) and the Ollama model definitions; consider moving these into configuration/env so they can be reused and adjusted without code changes and stay in sync with litellm/config.yaml.
• In lifespan, _register_ollama_models_in_db is always called regardless of whether litellm_master_key is set; adding an early return/log when the master key is missing would avoid unnecessary HTTP attempts and clearer behavior in misconfigured environments.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The new `_register_ollama_models_in_db` helper hardcodes `admin_url` (127.0.0.1:4000) and the Ollama model definitions; consider moving these into configuration/env so they can be reused and adjusted without code changes and stay in sync with `litellm/config.yaml`.
- In `lifespan`, `_register_ollama_models_in_db` is always called regardless of whether `litellm_master_key` is set; adding an early return/log when the master key is missing would avoid unnecessary HTTP attempts and clearer behavior in misconfigured environments.

## Individual Comments

### Comment 1
<location path="scripts/verification/verify_ollama_cooldown.py" line_range="26-35" />
<code_context>
-            for line in lines:
-                if line.startswith("triage_requests_total"):
-                    return int(float(line.split()[1]))
+        response = httpx.get(METRICS_URL, timeout=5.0)
+        lines = response.text.splitlines()
+        for line in lines:
+            if line.startswith("triage_requests_total"):
+                return int(float(line.split()[1]))
     except Exception as e:
         print(f"Error fetching metrics: {e}")
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Add response status checking when fetching metrics to avoid parsing error pages.

Because the code parses `response.text` for any status, a 4xx/5xx HTML/error body will cause the `triage_requests_total` parsing to fail and fall back to 0, masking real issues with the metrics endpoint. Consider calling `response.raise_for_status()` after the GET (inside the try) so failures surface explicitly while keeping the control flow the same.

```suggestion
def get_triage_request_count():
    try:
        response = httpx.get(METRICS_URL, timeout=5.0)
        response.raise_for_status()
        lines = response.text.splitlines()
        for line in lines:
            if line.startswith("triage_requests_total"):
                return int(float(line.split()[1]))
    except Exception as e:
        print(f"Error fetching metrics: {e}")
    return 0
```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}