Skip to content

feat: add upgrade-prod.sh for release-to-prod deployments #284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sheepdestroyer wants to merge 5 commits into from
Closed

feat: add upgrade-prod.sh for release-to-prod deployments #284

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1233f05
feat: add upgrade-prod.sh for syncing runtime files from releases
Jul 12, 2026
c9892a2
fix: address Gemini Code Assist review on upgrade-prod.sh
Jul 12, 2026
1000c62
fix: move upgrade-prod.sh to scripts/ per repo convention
Jul 12, 2026
a349e97
fix: bash syntax error in env var validation (Gemini review)
Jul 12, 2026
3e24a3e
fix: guard .env sourcing with set +u to prevent crashes from unbound …
Jul 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 30 additions & 9 deletions scripts/upgrade-prod.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ fi

# ── centralized cleanup ──
cleanup() {
rm -rf "${TEMP_DIR:-}"
rm -f "${UPGRADE_PROD_SELF_PATH:-}"
[[ -n "${TEMP_DIR:-}" ]] && rm -rf "$TEMP_DIR"
[[ -n "${UPGRADE_PROD_SELF_PATH:-}" ]] && rm -f "$UPGRADE_PROD_SELF_PATH"
}
trap cleanup EXIT

Expand Down Expand Up @@ -64,7 +64,7 @@ echo "🏷 Target release: $TAG"
TEMP_DIR=$(mktemp -d /tmp/llm-routing-upgrade.XXXXXX)

echo "📥 Cloning $REPO @ $TAG..."
git clone --depth 1 --branch "$TAG" "https://github.com/$REPO.git" "$TEMP_DIR" 2>&1 | tail -1
git clone -q --depth 1 --branch "$TAG" "https://github.com/$REPO.git" "$TEMP_DIR"

# ── verify the tag has the files we need ──
for f in pod.yaml start-stack.sh litellm/ router/ scripts/; do
Expand Down Expand Up @@ -104,20 +104,41 @@ else
echo "Non-interactive shell detected, proceeding with upgrade..."
fi

# ── pre-flight: validate PROD_DIR ──
# ── pre-flight: validate PROD_DIR and required env vars ──
if [ ! -f "$PROD_DIR/.env" ]; then
echo "❌ $PROD_DIR/.env not found. Is PROD_DIR correct?"
exit 1
fi

# ── stop the pod gracefully before touching files ──
POD_NAME="${POD_NAME:-agent-router-pod}"
if podman pod exists "$POD_NAME" 2>/dev/null; then
echo "🛑 Stopping $POD_NAME (SIGTERM, 30s)..."
podman pod stop -t 30 "$POD_NAME" 2>/dev/null || true
# Source .env to validate required vars are present
# Temporarily disable set -u: .env files may contain unbound variable references
# (e.g., FOO=$BAR where BAR is unset) that would crash with set -u active.
set +u
set -a; source "$PROD_DIR/.env"; set +a

@gemini-code-assist gemini-code-assist Bot Jul 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Since set -u is active in this script (set -euo pipefail on line 15), sourcing a .env file that contains any unbound variables (e.g., FOO=$BAR where BAR is unset, or conditional expansions without defaults) will cause the script to crash immediately. Since .env files are user-configured and may not be strictly compliant with set -u, it is safer to temporarily disable set -u while sourcing .env and re-enable it afterward.

Suggested change
set -a; source "$PROD_DIR/.env"; set +a
set +u
set -a; source "$PROD_DIR/.env"; set +a
set -u

set -u

missing_vars=()
for var in OPENROUTER_API_KEY OLLAMA_API_KEY LLAMA_CLASSIFIER_URL PUBLIC_BASE_URL; do
if [[ ! -v "$var" ]] || [[ -z "${!var}" ]]; then
missing_vars+=("$var")
fi
done

if [ ${#missing_vars[@]} -gt 0 ]; then
echo "❌ Missing required env vars in $PROD_DIR/.env:"
for var in "${missing_vars[@]}"; do
echo " - $var"
done
echo ""
echo "Add them before upgrading. Example:"
echo " echo '${missing_vars[0]}=your_value' >> $PROD_DIR/.env"
exit 1
fi

# ── rsync runtime files ──
# The pod stays running during rsync — it only uses rendered configs from
# DATA_ROOT, not the source files being synced. start-stack.sh --pull
# handles graceful shutdown (including pre-deploy backup) before redeploy.
echo "📋 Syncing runtime files..."
# Sync directories with --delete (clean stale files within each dir)
rsync -a --delete "$TEMP_DIR/litellm/" "$PROD_DIR/litellm/"
Expand Down