chore: restore reverted test suites, folder structure, logic optimizations, and security patches

.agents/AGENTS.md

@@ -18,3 +18,11 @@ When working on this project, always refer to the dedicated **NotebookLM Compani
 Use the `notebooklm` MCP tools to search or ask questions about this codebase and stack:
 - Run `notebook_ask` with `notebook_id: "llm-triage-gateway"` to ground your reasoning or implementation plans.
 - If you need session continuation, remember to reuse the `session_id` returned by previous queries.
+
+## Git Rebase & Conflict Resolution Policy
+To prevent directory reorganization regressions, outdated file restorations, or security credential overrides during merge conflict resolution, all automated agents must strictly follow these rules:
+
+1. **Rebase Over Merge**: Always fetch and rebase the topic/feature branch onto the latest `master` base branch (using `git rebase origin/master`) instead of performing `git merge`.
+2. **Directory Rename Safety**: If Git reports conflicts related to moved directories or files, do not manually stage deletions of folders (`tests/`, `scripts/`) or re-create files at the root level. Direct all changes to the newly refactored paths.
+3. **Verify Security Credentials**: Never accept resolutions that overwrite configuration files (`pod.yaml`, `start-stack.sh`) with hardcoded default passwords. Ensure placeholder-based configurations are preserved.
+4. **Enforce Test Suite Count**: Run the full unit test suite (`pytest`) after conflict resolution. Verify that the total number of passing tests is equal to or greater than before the resolution.

.agents/jules_regression_analysis.md

@@ -0,0 +1,81 @@
+# Analysis of Bot Merge Conflict Regressions & Guardrails
+
+This document breaks down the root causes of the regressions introduced by the automated agent (`jules[bot]`) during recent merges, compares Git resolution strategies, and outlines prompt modifications and guardrails to prevent future occurrences.
+
+---
+
+## 1. What Went Wrong?
+
+The primary failure arose from a combination of **reorganization conflict logic** and **blind resolution choices**:
+
+### A. The Directory Rename / Delete Conflict Mismatch
+* **Context**: PR #181 reorganized the repository by moving files at the root (like `test_a2_verify.py` and `host_agy_daemon.py`) into nested directories (`tests/` and `scripts/`).
+* **The Bot's Branch State**: The bot was working on older branches (`perf-optimize-gemini-oauth-token`, `refactor-record-tool-usage`) whose base commits predated the reorganization.
+* **The Failure**: When merging `master` into these older feature branches, Git detected conflict types like `rename/delete` or `directory rename detection`. Because the bot resolved conflicts locally inside the old workspace structure, it staged the **deletion** of the new files inside `tests/` and `scripts/` and re-introduced older, obsolete root-level files.
+* **Result**: Merges from these branches back to `master` cleanly deleted the subfolder-bound tests and re-added old copies at the root.
+
+### B. Lack of Cross-Check Verification
+* The bot resolved conflicts file-by-file without compiling the whole project or validating the full test suite (`pytest`) on the combined codebase.
+* It accepted obsolete file versions wholesale (e.g., reverting the dynamic passwords in `pod.yaml` and `start-stack.sh` to hardcoded credentials) because it assumed conflict markers in one block did not impact security/logic blocks elsewhere.
+
+---
+
+## 2. Which Git Strategy Should Have Been Used?
+
+Instead of merging `master` directly into older feature branches (which creates complex, multi-directional merge graphs), the following strategies are far safer for LLM agents:
+
+### Option A: `git rebase master` (Recommended for Feature Branches)
+Rather than a merge commit, the agent should rebase the feature branch onto the latest `master` commit:
+```bash
+git checkout feature-branch
+git fetch origin
+git rebase origin/master
+```
+* **Why it works**: Rebase replays each feature branch commit one-by-one on top of the reorganized `master` commit. 
+* **Rename Tracking**: Git's rename tracking algorithm handles moves seamlessly during a rebase. If a file was renamed from `test_a2_verify.py` to `tests/test_a2_verify.py` on `master`, Git will automatically apply the feature branch's modifications to `tests/test_a2_verify.py` instead of leaving them at the root.
+
+### Option B: Merge with explicit Merge Drivers
+If rebasing is not used, the agent must inspect renames before committing:
+```bash
+git diff --name-status origin/master...HEAD
+```
+This lists any deleted/added files to quickly verify that no folder moves were silently discarded.
+
+---
+
+## 3. Recommended Prompt Guardrails and System Rules
+
+To prevent automated agents from causing directory and code regressions, the following rules should be appended to the agent's instructions (e.g., in `.agents/AGENTS.md` or system guidelines):
+
+### Rule 1: Git Conflict Rebase Mandate
+> [!IMPORTANT]
+> When updating a feature branch with `master`/`main` changes, always prefer `git rebase` over `git merge`. If conflicts arise due to renamed directories, do not manually delete folders or re-add root counterparts. Ensure files are modified in their new paths.
+
+### Rule 2: Complete Test Suite Verification
+> [!WARNING]
+> Never push conflict resolutions without running the full test suite.
+> * If the workspace previously had $N$ passing tests, the resolved branch must have at least $N$ passing tests.
+> * Confirm all files staged for deletion or addition are intentional using `git diff --stat origin/master`.
+
+### Rule 3: File Integrity & Verification Checklist
+Add a post-conflict verification script step to the agent workflow:
+```bash
+# Verify no files were moved to root unexpectedly
+git status | grep -E "renamed:|deleted:|new file:"
+```
+
+---
+
+## 4. Proposed Instructions / Prompts for Bot Agents
+
+When tasking an agent with conflict resolution or merging, use a structured prompt like this:
+
+```markdown
+You are resolving merge conflicts for the branch [branch_name].
+
+1. Run `git fetch origin` and `git rebase origin/master`.
+2. If conflicts arise, inspect whether any files were renamed/moved in master. Apply your changes to the renamed files in their new locations, rather than re-creating them at their old locations.
+3. Once rebased, run the entire test suite: `pytest`.
+4. Run `git diff --stat origin/master` and review every file addition/deletion. Verify that no tests or scripts are missing compared to master.
+5. If any test files or core logic sections are missing, checkout the missing files from master and apply changes cleanly.
+```

.github/workflows/test.yml

@@ -23,13 +23,13 @@ jobs:
           python-version: '3.11'
 
       - name: Install dependencies
-        run: pip install httpx==0.28.1 pytest pytest-asyncio anyio pyyaml fastapi "pydantic>=2.0,<3.0" uvicorn python-multipart asyncpg langfuse redis
+        run: pip install httpx==0.28.1 pytest pytest-asyncio anyio pyyaml fastapi "pydantic>=2.0,<3.0" uvicorn python-multipart asyncpg langfuse redis aiofiles
 
       - name: Run Unit Tests
-        run: CONFIG_PATH=router/config.yaml PYTHONPATH=.:router pytest --ignore=test_agy_behavior.py --ignore=test_agy_tiers.py
+        run: CONFIG_PATH=router/config.yaml PYTHONPATH=.:router pytest --ignore=tests/test_agy_behavior.py --ignore=tests/test_agy_tiers.py
 
       - name: Run Breaker Verification
-        run: python3 verify_breaker.py
+        run: PYTHONPATH=. python3 scripts/verification/verify_breaker.py
 
       - name: Run Integration Verification
-        run: python3 test_a2_verify.py
+        run: PYTHONPATH=.:router python3 tests/test_a2_verify.py

README.md

@@ -66,7 +66,7 @@ graph TD
     style QwenLocal fill:#f0f0f0,stroke:#999,stroke-width:1px;
 ```
 
-> **Version Pin**: LiteLLM Gateway runs `ghcr.io/berriai/litellm:v1.88.0`. See §3B for pinning policy.
+> **Version Pin**: LiteLLM Gateway runs `ghcr.io/berriai/litellm:v1.90.2`. See §3B for pinning policy.
 
 ---
 
@@ -269,7 +269,7 @@ Exposes the entry endpoint (`http://localhost:5000/v1`) and evaluates prompt com
 > Model capabilities, token limits, and costs are visible in LiteLLM's Model Hub Table at `http://localhost:4000/ui/?page=model-hub-table` (or port 4000 on the gateway host).
 
 ### B. LiteLLM Proxy Gateway (`litellm/config.yaml`)
-- **Version Pinning**: The LiteLLM gateway runs `ghcr.io/berriai/litellm:v1.88.0` (latest stable as of June 2026). The tag is explicitly pinned in `pod.yaml` — never use `:latest`. Check available tags with `skopeo list-tags docker://ghcr.io/berriai/litellm` before upgrading. ClickHouse runs `docker.io/clickhouse/clickhouse-server:26.5.1` (upgraded from 24.8, June 2026). Valkey Cache runs `docker.io/valkey/valkey:9.1.0-alpine` (upgraded from 8, June 2026).
+- **Version Pinning**: The LiteLLM gateway runs `ghcr.io/berriai/litellm:v1.90.2` (latest stable as of June 2026). The tag is explicitly pinned in `pod.yaml` — never use `:latest`. Check available tags with `skopeo list-tags docker://ghcr.io/berriai/litellm` before upgrading. ClickHouse runs `docker.io/clickhouse/clickhouse-server:26.5.1` (upgraded from 24.8, June 2026). Valkey Cache runs `docker.io/valkey/valkey:9.1.0-alpine` (upgraded from 8, June 2026).
 Orchestrates routing fallback chains, Redis caching, and telemetry callbacks:
 - **`drop_params: true`**: Automatically strips unsupported arguments when transitioning to models that don't support them.
 - **Request Timeouts (`300s`)**: Provides ample padding to prevent connection aborts during dynamic RAM swapping operations on the local GPU `llama-server`.

pod.yaml

@@ -45,7 +45,7 @@ spec:
       value: sk-lit...33bf
     - name: OLLAMA_API_KEY
       value: 3cd542f8fca14fd08bedfd4f2ab36f9f.6G7Iukbvu1Keyi9x8eKckNEO
-    image: ghcr.io/berriai/litellm:v1.89.4
+    image: ghcr.io/berriai/litellm:v1.90.2
     livenessProbe:
       exec:
         command:
@@ -273,9 +273,9 @@ spec:
     - name: LANGFUSE_S3_EVENT_UPLOAD_REGION
       value: us-east-1
     - name: LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID
-      value: minioadmin
+      value: MINIO_USER_PLACEHOLDER
     - name: LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY
-      value: minioadmin
+      value: MINIO_PASSWORD_PLACEHOLDER
     - name: LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT
       value: http://127.0.0.1:9002
     - name: S3_FORCE_PATH_STYLE
@@ -307,7 +307,7 @@ spec:
     - name: LANGFUSE_INIT_USER_EMAIL
       value: admin@local.dev
     - name: LANGFUSE_INIT_USER_PASSWORD
-      value: admin-local-pw-2026
+      value: LANGFUSE_INIT_USER_PASSWORD_PLACEHOLDER
     - name: LANGFUSE_LOG_LEVEL
       value: warn
     image: docker.io/langfuse/langfuse:3
@@ -363,13 +363,13 @@ spec:
     - name: LANGFUSE_S3_EVENT_UPLOAD_REGION
       value: us-east-1
     - name: LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID
-      value: minioadmin
+      value: MINIO_USER_PLACEHOLDER
     - name: S3_FORCE_PATH_STYLE
       value: 'true'
     - name: LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT
       value: http://127.0.0.1:9002
     - name: LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY
-      value: minioadmin
+      value: MINIO_PASSWORD_PLACEHOLDER
     - name: LANGFUSE_LOG_LEVEL
       value: warn
     image: docker.io/langfuse/langfuse-worker:3
@@ -393,9 +393,9 @@ spec:
     - ":9001"
     env:
     - name: MINIO_ROOT_USER
-      value: minioadmin
+      value: MINIO_USER_PLACEHOLDER
     - name: MINIO_ROOT_PASSWORD
-      value: minioadmin
+      value: MINIO_PASSWORD_PLACEHOLDER
     image: docker.io/minio/minio:latest
     livenessProbe:
       httpGet:

router/Dockerfile

@@ -3,7 +3,7 @@ FROM python:3.14-slim
 WORKDIR /app
 
 # Install deps in a single layer (no pip cache, no extra files)
-RUN pip install --no-cache-dir fastapi "pydantic>=2.0,<3.0" uvicorn httpx pyyaml python-multipart asyncpg langfuse redis
+RUN pip install --no-cache-dir fastapi "pydantic>=2.0,<3.0" uvicorn httpx pyyaml python-multipart asyncpg langfuse redis aiofiles
 
 # Copy all source in one layer — removes dead config COPY (volume-mounted at runtime)
 COPY main.py agy_proxy.py circuit_breaker.py aa_scores.json free_models_roster.json /app/

router/main.py

@@ -1,4 +1,6 @@
 import os
+import aiofiles
+import re
 import sys
 import json
 import time
@@ -49,10 +51,15 @@ def get_redis():
             return None
         _redis_last_init_attempt = now
         try:
-            host = os.getenv("VALKEY_HOST", "127.0.0.1")
-            port = int(os.getenv("VALKEY_PORT", "6379"))
-            _redis_client = aioredis.Redis(host=host, port=port, decode_responses=True, socket_timeout=1.0)
-            logger.info(f"Valkey client initialized at {host}:{port}")
+            url = os.getenv("VALKEY_URL")
+            if url:
+                _redis_client = aioredis.Redis.from_url(url, decode_responses=True, socket_timeout=1.0)
+                logger.info("Valkey client initialized from URL")
+            else:
+                host = os.getenv("VALKEY_HOST", "127.0.0.1")
+                port = int(os.getenv("VALKEY_PORT", "6379"))
+                _redis_client = aioredis.Redis(host=host, port=port, decode_responses=True, socket_timeout=1.0)
+                logger.info(f"Valkey client initialized at {host}:{port}")
         except Exception as e:
             logger.warning(f"Failed to initialize Valkey client: {e} — falling back to local memory")
             _redis_client = None
@@ -82,24 +89,60 @@ def get_http_client():
     return _http_client
 
 
+# Compiled regular expressions for token estimation heuristics
+WORD_RE = re.compile(r'[a-zA-Z0-9]+')
+NON_ASCII_RE = re.compile(r'[^\s\x00-\x7F]')
+PUNC_RE = re.compile(r'[\x21-\x2f\x3a-\x40\x5b-\x60\x7b-\x7e]')
+
+
+def _count_tokens_heuristic(text: str) -> float:
+    """Heuristically estimate token count using weighted categories and optimized regex splitting.
+
+    This replaces the naive character-count logic with a more granular approach that
+    balances English words, technical identifiers, punctuation, and multi-byte characters.
+
+    Returns a float to prevent intermediate rounding errors when summing across multiple
+    message blocks. Callers should round the total sum to convert it to an integer.
+    """
+    if not text:
+        return 0.0
+
+    # 1. Alphanumeric runs (Words/Identifiers/Hashes/Base64)
+    # Use a length-aware heuristic to avoid under-counting technical content.
+    word_matches = WORD_RE.findall(text)
+    word_total = sum(1.2 if len(w) <= 8 else len(w) / 4.0 for w in word_matches)
+
+    # 2. Non-ASCII characters (CJK/Emoji)
+    # Each character is weighted at 0.35 tokens.
+    non_ascii_count = len(NON_ASCII_RE.findall(text))
+
+    # 3. ASCII Punctuation/Symbols
+    # Characters that are ASCII but not alphanumeric or whitespace.
+    punc_count = len(PUNC_RE.findall(text))
+
+    return word_total + (non_ascii_count * 0.35) + (punc_count * 0.4)
+
+
 def estimate_prompt_tokens(body: dict) -> int:
-    """Estimate prompt tokens by counting characters in message contents (1 token ~= 4 chars)
-    to avoid inflating metrics with large tool/schema declarations.
+    """Estimate prompt tokens using a regex-based weighted heuristic for mixed content.
     """
-    tokens = 0
+    total = 0.0
     for msg in body.get("messages", []):
         if not isinstance(msg, dict):
             continue
         content = msg.get("content") or ""
         if isinstance(content, str):
-            tokens += len(content) // 4
+            total += _count_tokens_heuristic(content)
         elif isinstance(content, list):
             for block in content:
                 if isinstance(block, dict) and block.get("type") == "text":
-                    tokens += len(block.get("text") or "") // 4
-    # Include a flat estimate for system prompt / metadata overhead
-    tokens += 50
-    return max(1, tokens)
+                    text = block.get("text")
+                    if isinstance(text, str):
+                        total += _count_tokens_heuristic(text)
+
+    # Include a flat estimate for system prompt / metadata overhead.
+    # Use rounding to avoid truncation bias (e.g., 1.9 -> 1).
+    return max(1, int(round(total)) + 50)
 
 
 async def sync_cooldowns_from_valkey() -> None:
@@ -3871,9 +3914,26 @@ def validate_payload(self) -> "AnnotationPayload":
 annotations_lock = asyncio.Lock()
 
 
-def _read_annotations_sync(path) -> dict:
-    with open(path, "r", encoding="utf-8") as f:
-        return json.load(f)
+_annotations_cache = {}
+
+
+async def _read_annotations_async(path) -> dict:
+    import copy
+
+    # Do not swallow OSError if file doesn't exist to preserve original behavior.
+    # The caller (save_annotations) handles the exception when reading existing annotations.
+    current_mtime = await asyncio.to_thread(os.path.getmtime, path)
+
+    cache_entry = _annotations_cache.get(path)
+
+    if cache_entry is None or current_mtime != cache_entry["mtime"]:
+        async with aiofiles.open(path, "r", encoding="utf-8") as f:
+            # Read asynchronously, but parse in a thread pool to avoid blocking event loop
+            content = await f.read()
+            data = await asyncio.to_thread(json.loads, content)
+            _annotations_cache[path] = {"mtime": current_mtime, "data": data}
+
+    return await asyncio.to_thread(copy.deepcopy, _annotations_cache[path]["data"])
 
 
 @app.post("/dashboard/save-annotations")
@@ -3887,9 +3947,7 @@ async def save_annotations(payload: AnnotationPayload):
         async with annotations_lock:
             if ann_path.exists():
                 try:
-                    existing = await asyncio.to_thread(
-                        _read_annotations_sync, str(ann_path)
-                    )
+                    existing = await _read_annotations_async(str(ann_path))
                 except Exception as read_err:
                     logger.warning(
                         f"Could not read existing annotations: {read_err}. Overwriting."

router/memory_mcp.py

@@ -17,6 +17,7 @@
 import time
 import hashlib
 import httpx
+import urllib.parse
 
 API_URL = "http://127.0.0.1:5000/v1/memory"
 PROTOCOL_VERSION = "2024-11-05"
@@ -44,7 +45,8 @@ def _make_key(category: str, is_global: bool, data: str) -> str:
     # BLAKE2b: SOTA crypto hash, stdlib, faster than MD5, deterministic across restarts.
     # Provides uniqueness within the same millisecond.
     h = hashlib.blake2b((data + str(ts)).encode("utf-8"), digest_size=HASH_DIGEST_SIZE).hexdigest()
-    return f"{PREFIX}:{scope}:{category}::{ts}:{h}"
+    safe_category = urllib.parse.quote(category)
+    return f"{PREFIX}:{scope}:{safe_category}::{ts}:{h}"
 
 
 def _parse_key(key: str):
@@ -53,7 +55,7 @@ def _parse_key(key: str):
         parts = key.split("::")
         prefix = parts[0].split(":")  # memory:{scope}:{category}
         scope = prefix[1] if len(prefix) > 1 else ""
-        category = prefix[2] if len(prefix) > 2 else ""
+        category = urllib.parse.unquote(prefix[2]) if len(prefix) > 2 else ""
         ts_hash = parts[1] if len(parts) > 1 else ""
         ts = ts_hash.split(":")[0] if ts_hash else ""
         return {"scope": scope, "category": category, "timestamp": ts}
@@ -68,6 +70,8 @@ def _parse_key(key: str):
 
 def _is_memory_key(key: str) -> bool:
     """Check if a key follows the memory:{scope}:{category}:: format."""
+    if not isinstance(key, str):
+        return False
     return key.startswith(f"{PREFIX}:")