fix: address PR#252 review comments — safe_pod_teardown, .env key strip, empty-string port handling

[sheepdestroyer]

Summary

Addresses all valid review comments from PR #252 (Gemini Code Assist + Sourcery), plus a README consistency fix.

Changes

Fix: safe_pod_teardown false timeout report (Gemini)

podman pod exists returns 0 for stopped pods too, so after a successful graceful stop the script falsely reported a timeout and force-removed. Now uses podman pod inspect --format '{{.State}}' to check actual running state.

Fix: .env key not stripped (Gemini)

line.partition("=") preserves spaces around the key. Added key = key.strip() so "KEY " doesn't fail to match in os.environ.get().

Fix: empty-string port handling (Gemini)

os.getenv('PORT', 'default') returns "" when the var is set but empty, producing broken URLs like http://127.0.0.1:. Replaced all 9 occurrences in router/main.py and 1 in litellm/entrypoint.py with os.getenv('PORT') or 'default'.

Docs: README health check table

Updated valkey probes from tcpSocket to valkey-cli ping (matching the actual pod.yaml), and postgres from pg_isready -U postgres to pg_isready -U postgres -p <port>.

False positives (replied, no code change)

• LiteLLM /ping is a valid health endpoint alongside /health/liveness
• The or chain in entrypoint.py (os.environ.get("LITELLM_PORT") or os.environ.get("PORT") or "4000") correctly handles empty strings

Files changed

• start-stack.sh — safe_pod_teardown fix
• litellm/entrypoint.py — key strip + empty-string port fix
• router/main.py — empty-string port fix (9 occurrences)
• README.md — health check table update

Summary by Sourcery

Align pod deployment, ports, and health checks with environment-driven configuration and fix teardown and env loading edge cases.

New Features:

• Add support for a --pull mode and configurable router image and ports in the stack startup script.
• Introduce dev-specific startup wrapper and .env.dev overlay for running a separate dev pod alongside prod.
• Render environment-specific LiteLLM and router configs and ClickHouse port override files into a data-root directory driven by DATA_ROOT.

Bug Fixes:

• Fix safe_pod_teardown to only force-remove pods that are still running instead of stopped.
• Ensure .env keys are stripped before use and avoid overriding existing environment variables when loading them.
• Handle configurable PORT, POSTGRES_PORT, LITELLM_PORT, and VALKEY ports safely, including empty-string and invalid values, across router, entrypoint, backup, and health-check scripts.

Enhancements:

• Parameterize pod name, ports, image, and data directories in pod.yaml and start-stack.sh to support reusable prod/dev setups.
• Update router to derive service URLs and dashboard ports from environment variables and centralize Valkey port resolution.
• Refine MinIO, Postgres, LiteLLM, and router health checks and readiness logic to use configured ports and pod name variables.

Documentation:

• Update README liveness/readiness probe descriptions for Valkey and Postgres to match the actual health check commands.

Tests:

• Extend router Redis/Valkey tests to cover invalid port handling and connection failure logging.

[sourcery-ai]

Reviewer's Guide

Refactors the Podman-based deployment stack to be fully port- and pod-name-configurable, adds dev overlay support and rendered configs, fixes safe pod teardown and health/port handling, and aligns probes and docs with the actual runtime configuration.

Sequence diagram for safe_pod_teardown and deploy_fresh_pod flow

sequenceDiagram
    participant user
    participant start_stack_sh
    participant podman

    user->>start_stack_sh: invoke_with_flag(--full_rebuild_or_--pull_or_--replace_or_none)
    start_stack_sh->>start_stack_sh: set PULL_MODE_FULL_REBUILD_REPLACE_MODE
    start_stack_sh->>podman: pod_exists(POD_NAME)
    alt pod_exists
        alt FULL_REBUILD
            start_stack_sh->>podman: podman_build(ROUTER_IMAGE)
            start_stack_sh->>start_stack_sh: safe_pod_teardown()
            start_stack_sh->>podman: pod_stop(POD_NAME)
            start_stack_sh->>podman: pod_inspect_state(POD_NAME)
            alt state_Running
                start_stack_sh->>podman: pod_rm_force(POD_NAME)
            else state_not_running
                start_stack_sh->>podman: pod_rm(POD_NAME)
            end
            start_stack_sh->>start_stack_sh: deploy_fresh_pod()
        else PULL_MODE
            start_stack_sh->>podman: podman_pull(ROUTER_IMAGE)
            start_stack_sh->>start_stack_sh: safe_pod_teardown()
            start_stack_sh->>start_stack_sh: deploy_fresh_pod()
        else REPLACE_MODE
            start_stack_sh->>start_stack_sh: safe_pod_teardown()
            start_stack_sh->>start_stack_sh: deploy_fresh_pod()
        else restart_only
            start_stack_sh->>podman: pod_restart(POD_NAME)
            start_stack_sh->>start_stack_sh: setup_minio_buckets()
            start_stack_sh->>start_stack_sh: verify_stack_health()
        end
    else pod_not_exists
        start_stack_sh->>start_stack_sh: cleanup_zombie_ports()
        start_stack_sh->>start_stack_sh: deploy_fresh_pod()
    end

    note over start_stack_sh: deploy_fresh_pod_calls\ngenerate_clickhouse_config_render_litellm_config_render_router_config_render_pod_yaml_setup_minio_buckets_verify_stack_health

Loading

File-Level Changes

Change	Details	Files
Make pod deployment, ports, images, and volumes configurable and reusable for prod/dev, including new pull mode and helper scripts.	Add --pull mode and distinguish local rebuilds from pulling GHCR images, driven by ROUTER_IMAGE Introduce POD_NAME, DATA_ROOT, and per-service port env vars with defaults, and export them for downstream tooling Generate ClickHouse port override XML and rendered LiteLLM/router configs under DATA_ROOT, and mount those into the pod Refactor pod.yaml placeholders to use POD_NAME/DATA_ROOT and per-service port placeholders, plus derive PROXY_BASE_URL from PUBLIC_BASE_URL Add start-dev.sh and .env.dev to run an isolated dev pod alongside prod	start-stack.sh pod.yaml router/config.yaml .gitignore start-dev.sh .env.dev
Improve pod lifecycle safety and database backup portability by using POD_NAME and explicit port-aware health checks.	Fix safe_pod_teardown to inspect pod state and only force-remove when still running, using POD_NAME instead of a hardcoded name Update health verification to use env-driven ports for LiteLLM, router, Postgres, and MinIO, and to reuse POD_NAME in podman exec calls Make backup.sh source .env/.env.dev, parameterize PostgreSQL container name and port, and use pg_isready/pg_dump with POSTGRES_PORT	start-stack.sh scripts/backup.sh
Make router and LiteLLM components robust to empty or malformed env vars, and align them with configurable ports and Valkey settings.	Change all router and LiteLLM URL constructions to use os.getenv(...) or defaults so empty PORT-like envs no longer yield invalid URLs Introduce _valkey_port helper in router/main.py to resolve Valkey port from VALKEY_CACHE_PORT/VALKEY_PORT with validation and logging Update router dashboard, health checks, and Redis client initialization to respect LITELLM_PORT, LANGFUSE_WEB_PORT, ROUTER_PORT, and Valkey ports Adjust litellm/config.yaml to use VALKEY_CACHE_PORT_PLACEHOLDER and ROUTER_PORT_PLACEHOLDER for cache and router connectivity	router/main.py router/tests/test_get_redis.py litellm/config.yaml
Harden LiteLLM entrypoint env handling for .env files and dynamic ports, and make Postgres readiness configurable.	Strip whitespace from .env keys, and use os.environ.setdefault so existing env vars override .env values Parameterize PostgreSQL readiness to use POSTGRES_PORT with validation and logging, instead of hardcoding 5432 Resolve LiteLLM listening port from LITELLM_PORT, then PORT, then default 4000 to avoid empty-string issues	litellm/entrypoint.py
Align documentation and probes with the new probe commands and port configurability.	Update valkey health probes in README to use valkey-cli ping (with auth for valkey-lf) instead of tcpSocket Document Postgres probes with explicit -p to match the port-parameterized readiness/liveness checks in pod.yaml	README.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Warning

Review limit reached

@sheepdestroyer, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 19 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5ad0e2cd-ff98-48dd-9332-1572115e4011

📥 Commits

Reviewing files that changed from the base of the PR and between 3c2296e and cdd3114.

📒 Files selected for processing (12)

• .env.dev
• .gitignore
• README.md
• litellm/config.yaml
• litellm/entrypoint.py
• pod.yaml
• router/config.yaml
• router/main.py
• router/tests/test_get_redis.py
• scripts/backup.sh
• start-dev.sh
• start-stack.sh

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/learn-deployment-guidelines

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• You now resolve several ports from environment in multiple Python call sites using repeated os.getenv(... ) or 'default'; consider centralizing this into small helper functions/constants (similar to _valkey_port) for LiteLLM, router, and Langfuse ports to keep behavior consistent and avoid drift.
• In start-stack.sh, the new deploy_fresh_pod/port-rendering logic assumes certain envs (.env, .env.dev) and defaults; it may be worth adding a brief sanity check (e.g., ensuring key ports are distinct and numeric) before generating config files to fail fast on misconfigured dev/prod overlays.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- You now resolve several ports from environment in multiple Python call sites using repeated `os.getenv(... ) or 'default'`; consider centralizing this into small helper functions/constants (similar to `_valkey_port`) for LiteLLM, router, and Langfuse ports to keep behavior consistent and avoid drift.
- In `start-stack.sh`, the new `deploy_fresh_pod`/port-rendering logic assumes certain envs (.env, .env.dev) and defaults; it may be worth adding a brief sanity check (e.g., ensuring key ports are distinct and numeric) before generating config files to fail fast on misconfigured dev/prod overlays.

## Individual Comments

### Comment 1
<location path="litellm/entrypoint.py" line_range="19-21" />
<code_context>
             line = line.strip()
             if line and not line.startswith("#") and "=" in line:
                 key, _, val = line.partition("=")
+                key = key.strip()
                 val = val.strip().strip('"').strip("'")
-                os.environ[key] = val
+                os.environ.setdefault(key, val)

 # Load Gemini OAuth token from credentials JSON
</code_context>
<issue_to_address>
**question (bug_risk):** Using `os.environ.setdefault` for `.env` loading prevents `.env` values from overriding existing env vars, which may be unexpected.

This change reverses the previous precedence: existing env vars (e.g., from the pod spec) now override `.env` values, instead of the other way around. If you still want `.env` to act as an override for local/dev setups, consider keeping direct assignment or documenting the new precedence clearly so it’s intentional and visible.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces support for dynamic port assignments and multi-environment configurations (such as development overlays) by replacing hardcoded ports, pod names, and image names with placeholders across configuration files, scripts, and pod.yaml. It also adds rendering logic for LiteLLM and Router configurations, updates health probes, and introduces a start-dev.sh wrapper. Feedback on the changes points out a high-severity issue in pod.yaml where ROUTER_PORT and LANGFUSE_WEB_PORT are not passed to the llm-triage-router container's environment, which causes the dashboard to fall back to default ports.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The environment variables ROUTER_PORT and LANGFUSE_WEB_PORT are used by the llm-triage-router application (in router/main.py) to display the correct ports on the dashboard and to perform health checks. However, they are not passed to the llm-triage-router container's environment in pod.yaml. This causes the dashboard to always fall back to the default ports (5000 and 3001) even when a custom port configuration (such as in .env.dev) is used.

Please add ROUTER_PORT and LANGFUSE_WEB_PORT to the container's environment variables.

    - name: LITELLM_PORT
      value: 'LITELLM_PORT_PLACEHOLDER'
    - name: VALKEY_CACHE_PORT
      value: 'VALKEY_CACHE_PORT_PLACEHOLDER'
    - name: ROUTER_PORT
      value: 'ROUTER_PORT_PLACEHOLDER'
    - name: LANGFUSE_WEB_PORT
      value: 'LANGFUSE_WEB_PORT_PLACEHOLDER'

[sheepdestroyer]

Good catch! Fixed: added ROUTER_PORT and LANGFUSE_WEB_PORT env vars to the llm-triage-router container in pod.yaml. The router uses these for dashboard display (lines 3879, 3917 in main.py) and health checks (line 2681). Without them, the dashboard always shows default ports even when custom ports are configured via .env.dev.

[sheepdestroyer]

Closing in favor of a fresh PR with all review fixes applied. See #254.