Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
7a9755b
chore: restore deleted test suites, project directory structure, and …
sheepdestroyer Jul 1, 2026
d239166
docs: update LiteLLM version pinning and test suite paths
sheepdestroyer Jul 1, 2026
4422f46
chore: add bot rebase/conflict guidelines and regression report to .a…
sheepdestroyer Jul 1, 2026
7c457f7
ci: add aiofiles to dependencies and update test paths
sheepdestroyer Jul 1, 2026
4d8cea0
test: wrap test_agy_behavior.py execution in main block to prevent im…
sheepdestroyer Jul 1, 2026
e45e7fc
chore: restore category sanitization security patch in memory_mcp.py …
sheepdestroyer Jul 1, 2026
36fa94a
chore: address review feedback, restore async agy log reading and cla…
sheepdestroyer Jul 1, 2026
6cdf604
chore: restore clean secret placeholders in pod.yaml/start-stack.sh a…
sheepdestroyer Jul 1, 2026
660811f
chore: add branch audit plan and restore offload-aa-scores performanc…
sheepdestroyer Jul 2, 2026
ad44754
chore: address CodeRabbit review feedback on PR #198
sheepdestroyer Jul 2, 2026
b0a8605
perf: optimize async log checking in agy_proxy to seek and read 1KB
sheepdestroyer Jul 2, 2026
186c9d4
chore: address new CodeRabbit and Gemini Code Assist reviews
sheepdestroyer Jul 2, 2026
e7792af
fix(chore): resolve broken script paths, sys.path imports and relativ…
sheepdestroyer Jul 2, 2026
ba7deff
fix(triage): truncate classifier prompt to 300 chars to avoid cache-r…
sheepdestroyer Jul 2, 2026
9314a5e
fix(triage): map virtual target_model llm-routing-agy to agent-advanc…
sheepdestroyer Jul 2, 2026
d45b951
docs: add section 9d documenting live stack tier testing script and s…
sheepdestroyer Jul 2, 2026
1963807
feat(triage): make classifier prompt truncation limit configurable vi…
sheepdestroyer Jul 2, 2026
a08d453
chore: address PR review feedback and clean up test boilerplate
sheepdestroyer Jul 2, 2026
2eae001
Address and resolve PR #204 review feedback
sheepdestroyer Jul 2, 2026
ee7584d
Fix bash syntax error: remove local outside function
sheepdestroyer Jul 2, 2026
5f86090
chore: address unaddressed PR 205 review findings
sheepdestroyer Jul 2, 2026
af6489c
chore: address PR 206 review feedback and implement declarative PGDAT…
sheepdestroyer Jul 2, 2026
a142d51
chore: address unaddressed PR review feedback and clean up tests/docs
sheepdestroyer Jul 3, 2026
fb70eed
chore: address unresolved PR 208 reviews and fix doc inconsistencies
sheepdestroyer Jul 8, 2026
3e67ee7
chore: address unresolved PR 210 reviews and code quality findings
sheepdestroyer Jul 8, 2026
4aeaccd
Update router/main.py
sheepdestroyer Jul 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .agents/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,11 @@ When working on this project, always refer to the dedicated **NotebookLM Compani
Use the `notebooklm` MCP tools to search or ask questions about this codebase and stack:
- Run `notebook_ask` with `notebook_id: "llm-triage-gateway"` to ground your reasoning or implementation plans.
- If you need session continuation, remember to reuse the `session_id` returned by previous queries.

## Git Rebase & Conflict Resolution Policy
To prevent directory reorganization regressions, outdated file restorations, or security credential overrides during merge conflict resolution, all automated agents must strictly follow these rules:

1. **Rebase Over Merge**: Always fetch and rebase the topic/feature branch onto the latest `master` base branch (using `git rebase origin/master`) instead of performing `git merge`.
2. **Directory Rename Safety**: If Git reports conflicts related to moved directories or files, do not manually stage deletions of tracked files from moved directories (e.g., under the old `tests/` or `scripts/` paths) or re-create files at the root level. Resolve conflicts by directing all changes and file operations to the newly refactored paths.
3. **Verify Security Credentials**: Never accept resolutions that overwrite configuration files (`pod.yaml`, `start-stack.sh`) with hardcoded default passwords. Ensure placeholder-based configurations are preserved.
4. **Enforce Test Suite Count**: Run the full unit test suite (`pytest`) after conflict resolution. Verify that the total number of passing tests is equal to or greater than before the resolution.
56 changes: 56 additions & 0 deletions .agents/branch_audit_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Git Merges & Regressions Audit Report

## Problem Overview
During the merging of concurrent pull requests, automated conflict-resolution bots (`jules[bot]`) silently discarded or reverted several code blocks and logic optimizations.

To ensure complete stability, we performed a systematic, automated three-dot diffing audit (`git diff CURRENT_BRANCH...TARGET_BRANCH`) comparing our current workspace against every active and merged remote branch in the repository.

---

## Branch-by-Branch Audit Checklist

| Branch Name / PR | Status | Notes / Discrepancies Checked |
| :--- | :---: | :--- |
| `origin/cleanup/remove-unused-get-live-gemini-oauth-token` (PR #195) | **CLEAN** | All changes present in current codebase. |
| `origin/perf-optimize-gemini-oauth-token-7488735575783948165` (PR #158) | **CLEAN** | All changes present in current codebase. |
| `origin/refactor-record-tool-usage-17779903941173107649` (PR #157) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/get-pr-status-tests-18330387033569416831` (PR #193) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/perf-async-annotations-4499495305587811104` (PR #172) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/add-gemini-oauth-tests-17524029845400706758` (PR #163) | **CLEAN** | All changes present in current codebase. |
| `origin/dependabot/docker/berriai/litellm-v1.90.2` (PR #192) | **CLEAN** | Diff showed `v1.90.2` LiteLLM image version bump, which is already applied. |
| `origin/security/fix-memory-mcp-category-injection-14795006529015952965` (PR #173) | **CLEAN** | All changes present in current codebase. |
| `origin/test/add-read-annotations-sync-tests-12043640228306609857` (PR #168) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/test-parse-key-parameterized-14651890987031951450` (PR #169) | **CLEAN** | All changes present in current codebase. |
| `origin/perf/async-quota-exhausted-4722057800691922723` (PR #156) | **CLEAN** | All changes present in current codebase. |
| `origin/perf-benchmark-classifier-14678555128983342452` (PR #155) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/add-test-redis-from-url-12746122485451637611` (PR #174) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/test-is-memory-key-14668590644102848561` (PR #164) | **CLEAN** | All changes present in current codebase. |
| `origin/⚡/optimize-agy-log-read-1519383298987985129` (PR #170) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/add-oauth-status-tests-15591368961301252072` (PR #166) | **CLEAN** | All changes present in current codebase. |
| `origin/chore/test-get-goose-sessions-16558526271462662412` (PR #165) | **CLEAN** | All changes present in current codebase. |
| `origin/fix-http-client-proliferation-11317864999360875839` | **CLEAN** | HTTP client unification is fully present in our workspace. |
| `origin/fix/test-record-tool-usage-3580129709200353766` | **CLEAN** | Root-level test files were reorganized to `/tests`, fully preserved. |
| `origin/fix/secrets-relocation` (PR #184) | **CLEAN** | Checked out placeholders and updated `start-stack.sh`/`pod.yaml` securely. |
| `origin/perf/offload-aa-scores-sync-16551127323385849872` (PR #98) | **RESTORED** | **Regression Found**: The asynchronous scores loading optimization was lost on master. Re-implemented and verified. |

---

## Detailed Regression Analysis & Fix: Offload AA Scores Loading

### The Regression
* **File:** `router/main.py`
* **Issue:** `_load_aa_scores()` (which does a blocking synchronous JSON disk read of `aa_scores.json`) was called directly at free-model/roster call sites, specifically `sync_adaptive_router_roster()` and `get_best_free_model()`, rather than inside `compute_free_model_score()`. The fix was to move `_load_aa_scores()` behind `await asyncio.to_thread(...)` and remove the direct synchronous call from the scoring path.
* **Merged Fix Lost:** The merged PR #98 had offloaded this operation to a background thread pool using `await asyncio.to_thread(_load_aa_scores)` outside the loops in `sync_adaptive_router_roster` and `get_best_free_model`, but this change was completely discarded during automated conflict resolution.

### The Resolution
1. **`router/main.py`**:
* Restored `await asyncio.to_thread(_load_aa_scores)` inside `sync_adaptive_router_roster()` and `get_best_free_model()` if the cache is not yet loaded.
* Removed the blocking `_load_aa_scores()` call from inside `compute_free_model_score()`.
2. **`tests/test_compute_free_model_score.py`**:
* Updated the test cases to explicitly call `router_main._load_aa_scores()` within the test setups to align with the asynchronous offloading.

---

## Verification Status
* **Test Suite:** Ran `PYTHONPATH=router:. pytest`
* **Result:** **189 tests passed** successfully (100% success rate).
81 changes: 81 additions & 0 deletions .agents/jules_regression_analysis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Analysis of Bot Merge Conflict Regressions & Guardrails

This document breaks down the root causes of the regressions introduced by the automated agent (`jules[bot]`) during recent merges, compares Git resolution strategies, and outlines prompt modifications and guardrails to prevent future occurrences.

---

## 1. What Went Wrong?

The primary failure arose from a combination of **reorganization conflict logic** and **blind resolution choices**:

### A. The Directory Rename / Delete Conflict Mismatch
* **Context**: PR #181 reorganized the repository by moving files at the root (like `test_a2_verify.py` and `host_agy_daemon.py`) into nested directories (`tests/` and `scripts/`).
* **The Bot's Branch State**: The bot was working on older branches (`perf-optimize-gemini-oauth-token`, `refactor-record-tool-usage`) whose base commits predated the reorganization.
* **The Failure**: When merging `master` into these older feature branches, Git detected conflict types like `rename/delete` or `directory rename detection`. Because the bot resolved conflicts locally inside the old workspace structure, it staged the **deletion** of the new files inside `tests/` and `scripts/` and re-introduced older, obsolete root-level files.
* **Result**: Merges from these branches back to `master` cleanly deleted the subfolder-bound tests and re-added old copies at the root.

### B. Lack of Cross-Check Verification
* The bot resolved conflicts file-by-file without compiling the whole project or validating the full test suite (`pytest`) on the combined codebase.
* It accepted obsolete file versions wholesale (e.g., reverting the dynamic passwords in `pod.yaml` and `start-stack.sh` to hardcoded credentials) because it assumed conflict markers in one block did not impact security/logic blocks elsewhere.

---

## 2. Which Git Strategy Should Have Been Used?

Instead of merging `master` directly into older feature branches (which creates complex, multi-directional merge graphs), the following strategies are far safer for LLM agents:

### Option A: `git rebase master` (Recommended for Feature Branches)
Rather than a merge commit, the agent should rebase the feature branch onto the latest `master` commit:
```bash
git checkout feature-branch
git fetch origin
git rebase origin/master
```
* **Why it works**: Rebase replays each feature branch commit one-by-one on top of the reorganized `master` commit.
* **Verify Relocated Files**: While Git's rename detection can assist with directory/file moves during rebase, it is not foolproof. The agent must manually verify relocated files (such as renamed test paths) after rebasing to ensure all changes landed in their correct new locations rather than being lost or orphaned.

### Option B: Merge with explicit Merge Drivers
If rebasing is not used, the agent must inspect renames before committing:
```bash
git diff --name-status origin/master...HEAD
```
This lists any deleted/added files to quickly verify that no folder moves were silently discarded.

---

## 3. Recommended Prompt Guardrails and System Rules

To prevent automated agents from causing directory and code regressions, the following rules should be appended to the agent's instructions (e.g., in `.agents/AGENTS.md` or system guidelines):

### Rule 1: Git Conflict Rebase Mandate
> [!IMPORTANT]
> When updating a feature branch with `master`/`main` changes, always prefer `git rebase` over `git merge`. If conflicts arise due to renamed directories, do not manually delete folders or re-add root counterparts. Ensure files are modified in their new paths.

### Rule 2: Complete Test Suite Verification
> [!WARNING]
> Never push conflict resolutions without running the full test suite.
> * If the workspace previously had $N$ passing tests, the resolved branch must have at least $N$ passing tests.
> * Confirm all files staged for deletion, addition, or rename are intentional by running `git diff --name-only origin/master...HEAD` and inspecting the content diffs of renamed or moved files.

### Rule 3: File Integrity & Verification Checklist
Add a post-conflict verification script step to the agent workflow:
```bash
# Verify no files were moved to root unexpectedly
git status --porcelain
```

---

## 4. Proposed Instructions / Prompts for Bot Agents

When tasking an agent with conflict resolution or merging, use a structured prompt like this:

```markdown
You are resolving merge conflicts for the branch [branch_name].

1. Run `git fetch origin` and `git rebase origin/master`.
2. If conflicts arise, inspect whether any files were renamed/moved in master. Apply your changes to the renamed files in their new locations, rather than re-creating them at their old locations.
3. Once rebased, run the entire test suite: `pytest`.
4. Run `git diff --name-only origin/master...HEAD` and inspect the full content diff for renamed, deleted, or restored files.
5. Only restore files from `master` after confirming the deletion was accidental; do not auto-checkout files that may have been intentionally renamed or removed.
```
8 changes: 4 additions & 4 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ jobs:
python-version: '3.11'

- name: Install dependencies
run: pip install httpx==0.28.1 pytest pytest-asyncio anyio pyyaml fastapi "pydantic>=2.0,<3.0" uvicorn python-multipart asyncpg langfuse redis
run: pip install httpx==0.28.1 pytest pytest-asyncio anyio pyyaml fastapi "pydantic>=2.0,<3.0" uvicorn python-multipart asyncpg langfuse redis aiofiles==25.1.0

- name: Run Unit Tests
run: CONFIG_PATH=router/config.yaml PYTHONPATH=.:router pytest --ignore=test_agy_behavior.py --ignore=test_agy_tiers.py
run: CONFIG_PATH=router/config.yaml PYTHONPATH=.:router pytest --ignore=tests/test_agy_behavior.py --ignore=tests/test_agy_tiers.py --ignore=tests/test_antigravity.py

- name: Run Breaker Verification
run: python3 verify_breaker.py
run: PYTHONPATH=. python3 scripts/verification/verify_breaker.py

- name: Run Integration Verification
run: python3 test_a2_verify.py
run: PYTHONPATH=.:router python3 tests/test_a2_verify.py
Loading