Skip to content

feat: add upgrade-prod.sh for release-to-prod deployments #282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sheepdestroyer wants to merge 3 commits into from
Closed

feat: add upgrade-prod.sh for release-to-prod deployments #282

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1233f05
feat: add upgrade-prod.sh for syncing runtime files from releases
Jul 12, 2026
c9892a2
fix: address Gemini Code Assist review on upgrade-prod.sh
Jul 12, 2026
1000c62
fix: move upgrade-prod.sh to scripts/ per repo convention
Jul 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions scripts/upgrade-prod.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ fi

# ── centralized cleanup ──
cleanup() {
rm -rf "${TEMP_DIR:-}"
rm -f "${UPGRADE_PROD_SELF_PATH:-}"
[[ -n "${TEMP_DIR:-}" ]] && rm -rf "$TEMP_DIR"
[[ -n "${UPGRADE_PROD_SELF_PATH:-}" ]] && rm -f "$UPGRADE_PROD_SELF_PATH"
}
trap cleanup EXIT

Expand Down Expand Up @@ -64,7 +64,7 @@ echo "🏷 Target release: $TAG"
TEMP_DIR=$(mktemp -d /tmp/llm-routing-upgrade.XXXXXX)

echo "📥 Cloning $REPO @ $TAG..."
git clone --depth 1 --branch "$TAG" "https://github.com/$REPO.git" "$TEMP_DIR" 2>&1 | tail -1
git clone -q --depth 1 --branch "$TAG" "https://github.com/$REPO.git" "$TEMP_DIR"

# ── verify the tag has the files we need ──
for f in pod.yaml start-stack.sh litellm/ router/ scripts/; do
Expand Down Expand Up @@ -104,20 +104,37 @@ else
echo "Non-interactive shell detected, proceeding with upgrade..."
fi

# ── pre-flight: validate PROD_DIR ──
# ── pre-flight: validate PROD_DIR and required env vars ──
if [ ! -f "$PROD_DIR/.env" ]; then
echo "❌ $PROD_DIR/.env not found. Is PROD_DIR correct?"
exit 1
fi

# ── stop the pod gracefully before touching files ──
POD_NAME="${POD_NAME:-agent-router-pod}"
if podman pod exists "$POD_NAME" 2>/dev/null; then
echo "🛑 Stopping $POD_NAME (SIGTERM, 30s)..."
podman pod stop -t 30 "$POD_NAME" 2>/dev/null || true
# Source .env to validate required vars are present
set -a; source "$PROD_DIR/.env"; set +a

missing_vars=()
for var in OPENROUTER_API_KEY OLLAMA_API_KEY LLAMA_CLASSIFIER_URL PUBLIC_BASE_URL; do
if [ -z "${!var:-}" ]; then
missing_vars+=("$var")
fi
done
Comment on lines +117 to +121

@gemini-code-assist gemini-code-assist Bot Jul 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Combining indirect expansion (${!var}) with parameter modifiers like :- results in a bad substitution syntax error in Bash. Furthermore, because set -u is enabled, attempting to expand an unset variable indirectly will cause the script to crash immediately. To safely check if the variable is set and non-empty under set -u, use the [[ -v "$var" ]] builtin to verify it exists before checking its value.

Suggested change
for var in OPENROUTER_API_KEY OLLAMA_API_KEY LLAMA_CLASSIFIER_URL PUBLIC_BASE_URL; do
if [ -z "${!var:-}" ]; then
missing_vars+=("$var")
fi
done
for var in OPENROUTER_API_KEY OLLAMA_API_KEY LLAMA_CLASSIFIER_URL PUBLIC_BASE_URL; do
if [[ ! -v "$var" ]] || [[ -z "${!var}" ]]; then
missing_vars+=("$var")
fi
done


if [ ${#missing_vars[@]} -gt 0 ]; then
echo "❌ Missing required env vars in $PROD_DIR/.env:"
for var in "${missing_vars[@]}"; do
echo " - $var"
done
echo ""
echo "Add them before upgrading. Example:"
echo " echo '$var=your_value' >> $PROD_DIR/.env"

@gemini-code-assist gemini-code-assist Bot Jul 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The variable $var here refers to the leaked loop variable from the validation loop on line 117, which will always evaluate to PUBLIC_BASE_URL regardless of which variable was actually missing. To provide a helpful and accurate example, reference the first missing variable using ${missing_vars[0]}.

Suggested change
echo " echo '$var=your_value' >> $PROD_DIR/.env"
echo " echo '${missing_vars[0]}=your_value' >> $PROD_DIR/.env"

exit 1
fi

# ── rsync runtime files ──
# The pod stays running during rsync — it only uses rendered configs from
# DATA_ROOT, not the source files being synced. start-stack.sh --pull
# handles graceful shutdown (including pre-deploy backup) before redeploy.
echo "📋 Syncing runtime files..."
# Sync directories with --delete (clean stale files within each dir)
rsync -a --delete "$TEMP_DIR/litellm/" "$PROD_DIR/litellm/"
Expand Down