Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e1d235f
⚡ improve prompt token estimation accuracy
google-labs-jules[bot] Jun 25, 2026
3ee04f5
Merge branch 'master' into ⚡-accurate-token-estimation-logic-65560357…
sheepdestroyer Jun 30, 2026
8946bcd
Merge branch 'master' into ⚡-accurate-token-estimation-logic-65560357…
sheepdestroyer Jun 30, 2026
e610e7d
⚡ address code review feedback for token estimation logic
google-labs-jules[bot] Jun 30, 2026
639376c
⚡ improve prompt token estimation accuracy
google-labs-jules[bot] Jun 30, 2026
2cc6741
⚡ improve prompt token estimation accuracy and address review feedback
google-labs-jules[bot] Jun 30, 2026
b6d5f6e
⚡ address code review findings for token estimation and dashboard met…
google-labs-jules[bot] Jun 30, 2026
250f89a
🧪 [testing improvement] Refined token estimation heuristic
google-labs-jules[bot] Jun 30, 2026
3b33476
🧪 [testing improvement] Refined token estimation heuristic (v2)
google-labs-jules[bot] Jun 30, 2026
882bd22
Merge branch 'master' into ⚡-accurate-token-estimation-logic-65560357…
sheepdestroyer Jun 30, 2026
d109020
docs: document regex-based token estimation logic and script
sheepdestroyer Jun 30, 2026
b900840
🧪 [testing improvement] Refined token estimation heuristic (v3)
google-labs-jules[bot] Jun 30, 2026
82f95a7
fix: address PR reviews by compiling regex, type-checking blocks, and…
sheepdestroyer Jun 30, 2026
476cfb6
fix: restore master's security configurations, test helpers, dependen…
sheepdestroyer Jun 30, 2026
207fbb9
style: revert unnecessary f-string additions in test files
sheepdestroyer Jun 30, 2026
2e64160
Merge branch 'master' into ⚡-accurate-token-estimation-logic-65560357…
sheepdestroyer Jun 30, 2026
2201dcc
fix: address PR reviews, restore NotebookLM doc, fix isolated test co…
sheepdestroyer Jun 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,8 @@ router/free_models_roster.json
# agent artifacts
.hermes/
.jules/
.local/
workdirs/
pr_description.txt

# Dataset work in progress
data/
.cache/
test_output*.log

35 changes: 15 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,15 +76,15 @@ All core containers are configured with **Kubernetes-style liveness and readines

| Container | Liveness Probe | Readiness Probe |
|:---|---:|---:|
| **valkey-cache** (9.1.0-alpine) | `tcpSocket` on port 6379 every 10s | Same, every 5s |
| **litellm-gateway** | Python `urllib` GET `/ping` (port 4000) every 15s | Python `urllib` GET `/health/readiness` (port 4000) every 10s |
| **llm-triage-router** | Python `urllib` GET `/metrics` (port 5000) every 15s | Same, every 10s |
| **valkey-cache** (9.1.0-alpine) | `valkey-cli PING` every 10s | `valkey-cli PING` every 5s |
| **litellm-gateway** | Python `urllib` GET `/health` (port 4000, accepts 200/401) every 15s | Same, every 10s |
| **llm-triage-router** | Python `urllib` GET `/dashboard` (port 5000) every 15s | Same, every 10s |
| **postgres-db** | `pg_isready -U postgres` every 10s | Same, every 5s |
| **clickhouse-db** | `clickhouse-client --user clickhouse --password clickhouse --query "SELECT 1"` every 15s | `clickhouse-client --query "SELECT 1"` every 10s |
| **valkey-lf** (9.1.0-alpine) | `tcpSocket` on port 6380 every 10s | Same, every 5s |
| **langfuse-web** | `wget` GET `/api/health` (port 3001) every 15s | Same, every 10s |
| **langfuse-worker** | `pgrep node` every 15s | — |
| **minio-s3** | `httpGet` `/minio/health/live` (port 9002) every 15s | `httpGet` `/minio/health/ready` (port 9002) every 10s |
| **clickhouse-db** | `clickhouse-client --user clickhouse --password clickhouse --query "SELECT 1"` every 15s | Same, every 10s |
| **valkey-lf** | `redis-cli -p 6380 -a langfuse-redis-2026 PING` every 10s | Same, every 5s |
| **langfuse-web** | `wget -qO /dev/null http://127.0.0.1:3001/` every 15s | Same, every 10s |
| **langfuse-worker** | `pgrep -f langfuse-worker` every 15s | — |
| **minio-s3** | TCP socket check on port 9002 every 15s | Same, every 10s |

The pod-level `restartPolicy: Always` combined with these probes means Podman will restart any container that fails its health check or exits unexpectedly, enabling true self-healing for the entire stack.

Expand Down Expand Up @@ -213,7 +213,7 @@ All configurations, automation scripts, and databases are self-contained within

```
/home/gpav/Vrac/LAB/AI/LLM-Routing/
├── .env # Environment file for API keys, passwords, and generated secrets (ignored by git)
├── .env # Environment file for OpenRouter API Key (ignored by git)
├── .gitignore # Git ignore policy protecting secrets & database files
├── README.md # In-depth system and operational guide
├── pod.yaml # Podman Kubernetes template defining the 10-container stack
Expand Down Expand Up @@ -379,7 +379,7 @@ Run the startup script from the root of the repository:
# health probes, env vars, containers — no rebuild)
./start-stack.sh --full-rebuild # Full reset: rebuild image + recreate pod
```
*Note: If running for the first time, the script will prompt you for your `OpenRouter API Key`, securely saving it inside `.env` with restrictive permissions (`chmod 600`). The script also automatically generates and persists secure random secrets (`LITELLM_MASTER_KEY`, `POSTGRES_PASSWORD`, `NEXTAUTH_SECRET`, `SALT`, `ENCRYPTION_KEY`, and `ROUTER_API_KEY`) to this file on startup if they are missing.*
*Note: If running for the first time, the script will prompt you for your `OpenRouter API Key`, securely saving it inside `.env` with restrictive permissions (`chmod 600`).*

### 2. Verify Container Status
Check that all **10 containers** inside `agent-router-pod` are up and running:
Expand Down Expand Up @@ -583,17 +583,11 @@ Minio runs on ports **9001** (web console) and **9002** (S3 API). Credentials: `

### Health Check

MinIO's health is monitored using its native structured endpoints `/minio/health/live` (liveness) and `/minio/health/ready` (readiness) on port 9002:
Minio's minimal Go image has no HTTP client tools. The probe uses a raw TCP socket check:

```yaml
livenessProbe:
httpGet:
path: /minio/health/live
port: 9002
readinessProbe:
httpGet:
path: /minio/health/ready
port: 9002
exec:
command: [sh, -c, "exec 3<>/dev/tcp/127.0.0.1/9002 && echo ok"]
```

---
Expand Down Expand Up @@ -797,7 +791,8 @@ For auto-routing modes, the Triage Router handles failures by silently falling b

This project is supported by a dedicated NotebookLM companion notebook:
* **Notebook Name:** `TriageGate-Architect-KB`
* **Notebook ID:** llm-triage-gateway
* **Notebook ID:** `llm-triage-gateway`
* **URL:** [TriageGate-Architect-KB](https://notebooklm.google.com/notebook/826cbd87-7969-4b0e-a38e-5517b5ab7d28)

This notebook contains a comprehensive semantic index of the system architecture, LiteLLM cascades, Langfuse telemetry pipelines, local model configurations, and integration guides. Agents and developers can query this notebook via the `notebooklm` MCP tools (e.g., using `notebook_ask` with `notebook_id: "llm-triage-gateway"`) to retrieve structured knowledge, check pitfalls, or get implementation examples for this gateway stack.

10 changes: 4 additions & 6 deletions get_pr_status.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
import subprocess
from typing import Sequence
import shlex


def run_cmd(argv: Sequence[str]) -> str:
def run_cmd(cmd):
# Fix the issues from Sourcery review!
# 1. Provide a static list of strings for args rather than a single string.
# 2. Use shell=False
# 3. Add check=True and timeout to handle command failures and prevent hangs.
result = subprocess.run(argv, shell=False, capture_output=True, text=True, check=True, timeout=30)
args = shlex.split(cmd)
result = subprocess.run(args, shell=False, capture_output=True, text=True, check=True, timeout=30)
return result.stdout.strip()

Loading