Skip to content

🧹 [code health improvement] Refactor record_tool_usage to use dataclass#157

Merged
sheepdestroyer merged 13 commits into
masterfrom
refactor-record-tool-usage-17779903941173107649
Jul 1, 2026
Merged

🧹 [code health improvement] Refactor record_tool_usage to use dataclass#157
sheepdestroyer merged 13 commits into
masterfrom
refactor-record-tool-usage-17779903941173107649

Conversation

@sheepdestroyer

@sheepdestroyer sheepdestroyer commented Jun 30, 2026

Copy link
Copy Markdown
Owner

🎯 What:
Refactored the record_tool_usage function in router/main.py to accept a single ToolUsageRecord dataclass instance instead of 6 individual positional arguments. Updated all 4 call sites in the application and test files.

💡 Why:
Passing a high number of arguments to a function reduces maintainability and readability. Grouping these parameters into a single ToolUsageRecord dataclass makes the function signature cleaner, ensures type safety, and makes it much easier to extend the tracked metrics in the future without modifying every call site's signature.

Verification:

  • Ran the syntax checker (python3 -m py_compile router/main.py test_record_tool_usage.py) to confirm zero syntax errors.
  • Executed the full test suite (PYTHONPATH=$(pwd):$(pwd)/router CONFIG_PATH=router/config.yaml python3 -m pytest --import-mode=importlib) and confirmed all 111 tests passed successfully, ensuring no logic regressions were introduced during the refactor.

Result:
The record_tool_usage function is now cleaner, more maintainable, and utilizes a robust dataclass structure for tracking active tool metrics.


PR created automatically by Jules for task 17779903941173107649 started by @sheepdestroyer

Summary by Sourcery

Refactor tool usage tracking to use a dataclass-based record object for cleaner, more maintainable metrics collection.

Enhancements:

  • Introduce a ToolUsageRecord dataclass and update record_tool_usage to accept a single structured argument instead of multiple primitives.
  • Adjust router logic to construct and pass ToolUsageRecord instances at all tool usage recording call sites.

Tests:

  • Update record_tool_usage unit tests to construct ToolUsageRecord objects and validate that existing metrics and timeline behavior remain unchanged.

Summary by CodeRabbit

  • Refactor
    • Streamlined tool-usage tracking to use a single structured record, improving consistency across routing paths.
  • Bug Fixes
    • Made usage reporting more reliable for streamed and non-streamed requests, helping keep token counts, latency, and route metrics accurate.
    • Preserved timeline limits and background persistence behavior while updating the tracking flow.

@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@sourcery-ai

sourcery-ai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Refactors the tool usage tracking helper to use a single ToolUsageRecord dataclass parameter instead of multiple positional arguments, updating its implementation and all application and test call sites accordingly.

File-Level Changes

Change Details Files
Introduce ToolUsageRecord dataclass and update record_tool_usage implementation to consume it.
  • Add a ToolUsageRecord dataclass encapsulating tool name, token counts, model, latency, and route with a default route value.
  • Change record_tool_usage signature to accept a single ToolUsageRecord instance instead of multiple primitive arguments.
  • Update internal logic in record_tool_usage to reference fields from the dataclass instance, including mapping 'none' to 'other', computing totals, updating stats, and constructing the event payload.
router/main.py
Update all production call sites to construct and pass ToolUsageRecord instances.
  • Wrap existing record_tool_usage argument lists in ToolUsageRecord(...) calls within native_agy_stream_generator for both success paths.
  • Update stream_generator code paths to build ToolUsageRecord objects using existing variables (active_tool, token counts, model_name, latency, route) and pass them to record_tool_usage.
  • Ensure routing path values (google_oauth_direct, litellm_fallback) are preserved when creating ToolUsageRecord instances.
router/main.py
Adjust unit tests to work with the new ToolUsageRecord-based API.
  • Import ToolUsageRecord into the test module for use in constructing usage records.
  • Wrap existing record_tool_usage calls in tests with ToolUsageRecord(...) while preserving argument values and order.
  • Confirm that behavioral expectations (token accumulation, 'none' mapping, timeline size limit, custom route accounting) remain unchanged under the new API.
test_record_tool_usage.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@sheepdestroyer, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 15bc24fc-8a70-43ea-affb-012f75d6e998

📥 Commits

Reviewing files that changed from the base of the PR and between 150f447 and 578ba09.

📒 Files selected for processing (22)
  • .github/workflows/test.yml
  • .gitignore
  • README.md
  • get_pr_status.py
  • pod.yaml
  • router/Dockerfile
  • router/main.py
  • router/memory_mcp.py
  • router/test_memory_mcp.py
  • router/tests/test_detect_active_tool.py
  • router/tests/test_estimate_prompt_tokens.py
  • router/tests/test_get_goose_sessions.py
  • router/tests/test_get_live_gemini_oauth_token.py
  • router/tests/test_get_redis.py
  • router/tests/test_load_persisted_stats.py
  • router/tests/test_memory_mcp.py
  • scripts/README.md
  • scripts/benchmark_tokens.py
  • start-stack.sh
  • test_host_agy_daemon.py
  • test_pie_chart_gradient.py
  • tests/test_read_annotations_async.py
📝 Walkthrough

Walkthrough

A ToolUsageRecord dataclass is introduced in router/main.py, and record_tool_usage is refactored to accept this single structured object instead of separate scalar parameters. All call sites (agy streaming/non-streaming, LiteLLM streaming/non-streaming) and corresponding tests are updated accordingly.

Changes

ToolUsageRecord Refactor

Layer / File(s) Summary
ToolUsageRecord dataclass and core function
router/main.py
Adds a ToolUsageRecord dataclass (tool_name, prompt_tokens, completion_tokens, model, latency_ms, route) and refactors record_tool_usage(usage: ToolUsageRecord) to read fields from it while preserving stats/timeline persistence and throttled fallback behavior.
Call site updates
router/main.py
Updates agy native streaming, agy non-streaming, LiteLLM streaming, and LiteLLM non-streaming completion paths to construct ToolUsageRecord instances (with appropriate route values) and pass them to record_tool_usage, alongside a minor formatting tweak to avg_proxy_latency_ms.
Test updates
test_record_tool_usage.py
Imports ToolUsageRecord and refactors all record_tool_usage test invocations across basic, none-mapping, accumulation, timeline-limit, and custom-route tests to construct ToolUsageRecord instances instead of passing scalar arguments.

Estimated code review effort: 2 (Simple) | ~12 minutes

Possibly related PRs

  • sheepdestroyer/LLM-Routing#91: Adds/refactors the same test_record_tool_usage.py suite for router.main.record_tool_usage, the exact unit modified in this PR.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: refactoring record_tool_usage to use a dataclass.
Docstring Coverage ✅ Passed Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor-record-tool-usage-17779903941173107649

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • The initialization of stats['routing_paths'] can be simplified by using stats.setdefault('routing_paths', {'google_oauth_direct': 0, 'litellm_fallback': 0}) to avoid the explicit if check and keep the code more concise.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The initialization of `stats['routing_paths']` can be simplified by using `stats.setdefault('routing_paths', {'google_oauth_direct': 0, 'litellm_fallback': 0})` to avoid the explicit `if` check and keep the code more concise.

## Individual Comments

### Comment 1
<location path="router/main.py" line_range="1214-1215" />
<code_context>
-    latency_ms: float,
-    route: str = "litellm_fallback",
-):
+@dataclass
+class ToolUsageRecord:
+    tool_name: str
+    prompt_tokens: int
</code_context>
<issue_to_address>
**issue (bug_risk):** Avoid mutating fields on the ToolUsageRecord instance inside record_tool_usage.

Inside `record_tool_usage`, `usage.tool_name` is reassigned from "none" to "other". Mutating a dataclass instance provided by the caller can cause subtle bugs if the caller reuses it or assumes it is unchanged. Instead, derive a local variable for downstream logic (e.g. `tool_name = "other" if usage.tool_name == "none" else usage.tool_name`) and keep `ToolUsageRecord` unchanged.
</issue_to_address>

### Comment 2
<location path="router/main.py" line_range="1990-1997" />
<code_context>
                                 approx_prompt_tokens = estimate_prompt_tokens(body)

-                                record_tool_usage(
+                                record_tool_usage(ToolUsageRecord(
                                     active_tool,
                                     approx_prompt_tokens,
                                     token_count,
                                     model_name,
                                     latency_ms,
                                     route="google_oauth_direct",
-                                )
+                                ))
                                 logger.info(
                                     f"✅ native agy stream succeeded: {model_name}, {latency_ms:.0f}ms"
</code_context>
<issue_to_address>
**suggestion:** Positional construction of ToolUsageRecord is fragile if the dataclass evolves.

These call sites all construct `ToolUsageRecord` positionally, so any future change to the dataclass’s field order or additions could cause silent mis-logging. Please switch to keyword-based construction (e.g. `ToolUsageRecord(tool_name=..., prompt_tokens=..., ...)`) to make these calls resilient and more readable.

Suggested implementation:

```python
                                record_tool_usage(
                                    ToolUsageRecord(
                                        tool_name=active_tool,
                                        prompt_tokens=approx_prompt_tokens,
                                        completion_tokens=token_count,
                                        model_name=model_name,
                                        latency_ms=latency_ms,
                                        route="google_oauth_direct",
                                    )
                                )

```

```python
                        record_tool_usage(
                            ToolUsageRecord(
                                tool_name=active_tool,
                                prompt_tokens=prompt_tokens,
                                completion_tokens=completion_tokens,
                                model_name=model_name,

```

I can only see part of the second `ToolUsageRecord` call; you should:
1. Ensure any remaining positional arguments on that call (e.g. `latency_ms`, `route`, etc. if present later in the file) are also converted to keyword arguments consistent with the `ToolUsageRecord` dataclass definition.
2. Verify the keyword names (`tool_name`, `prompt_tokens`, `completion_tokens`, `model_name`, `latency_ms`, `route`) match the actual field names in `ToolUsageRecord`; adjust them if your dataclass uses different identifiers.
</issue_to_address>

### Comment 3
<location path="test_record_tool_usage.py" line_range="60-66" />
<code_context>
     """Test that tokens accumulate correctly over multiple calls."""
-    router.main.record_tool_usage("write", 10, 10, "model1", 50.0)
-    router.main.record_tool_usage("write", 20, 30, "model2", 60.0)
+    router.main.record_tool_usage(ToolUsageRecord("write", 10, 10, "model1", 50.0))
+    router.main.record_tool_usage(ToolUsageRecord("write", 20, 30, "model2", 60.0))

     assert router.main.stats["tool_tokens"]["write"] == 70
</code_context>
<issue_to_address>
**suggestion:** Prefer keyword arguments when constructing `ToolUsageRecord` in tests for robustness and readability.

These tests now depend on the positional field order in `ToolUsageRecord`. Using keyword arguments (e.g., `ToolUsageRecord(tool_name="write", prompt_tokens=10, ...)`) keeps the intent clear and avoids brittleness if field ordering changes while names stay the same, which better matches the maintainability goals of this PR.

```suggestion
def test_record_tool_usage_accumulation():
    """Test that tokens accumulate correctly over multiple calls."""
    router.main.record_tool_usage(
        ToolUsageRecord(
            tool_name="write",
            prompt_tokens=10,
            completion_tokens=10,
            model="model1",
            cost=50.0,
        )
    )
    router.main.record_tool_usage(
        ToolUsageRecord(
            tool_name="write",
            prompt_tokens=20,
            completion_tokens=30,
            model="model2",
            cost=60.0,
        )
    )

    assert router.main.stats["tool_tokens"]["write"] == 70
    assert router.main.stats["prompt_tokens"] == 30
    """Test that the timeline buffer is capped at 15 events."""
    # Add 20 events
    for i in range(20):
        router.main.record_tool_usage(
            ToolUsageRecord(
                tool_name=f"tool_{i}",
                prompt_tokens=1,
                completion_tokens=1,
                model="model",
                cost=10.0,
            )
        )
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread router/main.py
Comment on lines +1214 to +1215
@dataclass
class ToolUsageRecord:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Avoid mutating fields on the ToolUsageRecord instance inside record_tool_usage.

Inside record_tool_usage, usage.tool_name is reassigned from "none" to "other". Mutating a dataclass instance provided by the caller can cause subtle bugs if the caller reuses it or assumes it is unchanged. Instead, derive a local variable for downstream logic (e.g. tool_name = "other" if usage.tool_name == "none" else usage.tool_name) and keep ToolUsageRecord unchanged.

Comment thread router/main.py
Comment on lines +1990 to +1997
record_tool_usage(ToolUsageRecord(
active_tool,
approx_prompt_tokens,
token_count,
model_name,
latency_ms,
route="google_oauth_direct",
)
))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Positional construction of ToolUsageRecord is fragile if the dataclass evolves.

These call sites all construct ToolUsageRecord positionally, so any future change to the dataclass’s field order or additions could cause silent mis-logging. Please switch to keyword-based construction (e.g. ToolUsageRecord(tool_name=..., prompt_tokens=..., ...)) to make these calls resilient and more readable.

Suggested implementation:

                                record_tool_usage(
                                    ToolUsageRecord(
                                        tool_name=active_tool,
                                        prompt_tokens=approx_prompt_tokens,
                                        completion_tokens=token_count,
                                        model_name=model_name,
                                        latency_ms=latency_ms,
                                        route="google_oauth_direct",
                                    )
                                )
                        record_tool_usage(
                            ToolUsageRecord(
                                tool_name=active_tool,
                                prompt_tokens=prompt_tokens,
                                completion_tokens=completion_tokens,
                                model_name=model_name,

I can only see part of the second ToolUsageRecord call; you should:

  1. Ensure any remaining positional arguments on that call (e.g. latency_ms, route, etc. if present later in the file) are also converted to keyword arguments consistent with the ToolUsageRecord dataclass definition.
  2. Verify the keyword names (tool_name, prompt_tokens, completion_tokens, model_name, latency_ms, route) match the actual field names in ToolUsageRecord; adjust them if your dataclass uses different identifiers.

Comment thread test_record_tool_usage.py
Comment on lines 60 to 66
def test_record_tool_usage_accumulation():
"""Test that tokens accumulate correctly over multiple calls."""
router.main.record_tool_usage("write", 10, 10, "model1", 50.0)
router.main.record_tool_usage("write", 20, 30, "model2", 60.0)
router.main.record_tool_usage(ToolUsageRecord("write", 10, 10, "model1", 50.0))
router.main.record_tool_usage(ToolUsageRecord("write", 20, 30, "model2", 60.0))

assert router.main.stats["tool_tokens"]["write"] == 70
assert router.main.stats["prompt_tokens"] == 30

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Prefer keyword arguments when constructing ToolUsageRecord in tests for robustness and readability.

These tests now depend on the positional field order in ToolUsageRecord. Using keyword arguments (e.g., ToolUsageRecord(tool_name="write", prompt_tokens=10, ...)) keeps the intent clear and avoids brittleness if field ordering changes while names stay the same, which better matches the maintainability goals of this PR.

Suggested change
def test_record_tool_usage_accumulation():
"""Test that tokens accumulate correctly over multiple calls."""
router.main.record_tool_usage("write", 10, 10, "model1", 50.0)
router.main.record_tool_usage("write", 20, 30, "model2", 60.0)
router.main.record_tool_usage(ToolUsageRecord("write", 10, 10, "model1", 50.0))
router.main.record_tool_usage(ToolUsageRecord("write", 20, 30, "model2", 60.0))
assert router.main.stats["tool_tokens"]["write"] == 70
assert router.main.stats["prompt_tokens"] == 30
def test_record_tool_usage_accumulation():
"""Test that tokens accumulate correctly over multiple calls."""
router.main.record_tool_usage(
ToolUsageRecord(
tool_name="write",
prompt_tokens=10,
completion_tokens=10,
model="model1",
cost=50.0,
)
)
router.main.record_tool_usage(
ToolUsageRecord(
tool_name="write",
prompt_tokens=20,
completion_tokens=30,
model="model2",
cost=60.0,
)
)
assert router.main.stats["tool_tokens"]["write"] == 70
assert router.main.stats["prompt_tokens"] == 30
"""Test that the timeline buffer is capped at 15 events."""
# Add 20 events
for i in range(20):
router.main.record_tool_usage(
ToolUsageRecord(
tool_name=f"tool_{i}",
prompt_tokens=1,
completion_tokens=1,
model="model",
cost=10.0,
)
)

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules
There are unaddressed Reviews & Comments on this PR#
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate.
Make sure you did not miss any reviews or comments.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

waiting to retrigger gemini review

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve conflicts before continuing.

@jules

  • There are unaddressed Reviews & Comments on this PR#
    Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate.
    Make sure you did not miss any reviews or comments.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve conflicts before continuing.

1 similar comment
@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve all conflicts before continuing.

1 similar comment
@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve all conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@gemini review
@coderabbitai review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the record_tool_usage function in router/main.py to accept a new ToolUsageRecord dataclass instead of multiple individual arguments, updating its usages and corresponding tests accordingly. Feedback was provided regarding an anti-pattern where the input usage object is mutated in-place; it is recommended to use a local variable for the normalized tool name to avoid unexpected side effects.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread router/main.py
Comment on lines +1231 to 1254
if usage.tool_name == "none":
usage.tool_name = "other"

total = prompt_tokens + completion_tokens
stats["tool_tokens"][tool_name] = stats["tool_tokens"].get(tool_name, 0) + total
total = usage.prompt_tokens + usage.completion_tokens
stats["tool_tokens"][usage.tool_name] = stats["tool_tokens"].get(usage.tool_name, 0) + total

# Save global prompt/completion metrics
stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + prompt_tokens
stats["completion_tokens"] = stats.get("completion_tokens", 0) + completion_tokens
stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + usage.prompt_tokens
stats["completion_tokens"] = stats.get("completion_tokens", 0) + usage.completion_tokens

# Track routing path distribution
if "routing_paths" not in stats:
stats["routing_paths"] = {"google_oauth_direct": 0, "litellm_fallback": 0}
stats["routing_paths"][route] = stats["routing_paths"].get(route, 0) + 1
stats["routing_paths"][usage.route] = stats["routing_paths"].get(usage.route, 0) + 1

# Append to timeline event stack (in-memory ring buffer + persistent disk backup)
event = {
"timestamp": time.strftime("%H:%M:%S"),
"tool": tool_name,
"model": model,
"route": route,
"tool": usage.tool_name,
"model": usage.model,
"route": usage.route,
"tokens": total,
"latency_ms": int(latency_ms),
"latency_ms": int(usage.latency_ms),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Mutating the input parameter usage (specifically usage.tool_name) is an anti-pattern that can lead to unexpected side effects if the caller retains a reference to the ToolUsageRecord instance. Instead of modifying the object in-place, assign the normalized tool name to a local variable and use it throughout the function.

    tool_name = "other" if usage.tool_name == "none" else usage.tool_name

    total = usage.prompt_tokens + usage.completion_tokens
    stats["tool_tokens"][tool_name] = stats["tool_tokens"].get(tool_name, 0) + total

    # Save global prompt/completion metrics
    stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + usage.prompt_tokens
    stats["completion_tokens"] = stats.get("completion_tokens", 0) + usage.completion_tokens

    # Track routing path distribution
    if "routing_paths" not in stats:
        stats["routing_paths"] = {"google_oauth_direct": 0, "litellm_fallback": 0}
    stats["routing_paths"][usage.route] = stats["routing_paths"].get(usage.route, 0) + 1

    # Append to timeline event stack (in-memory ring buffer + persistent disk backup)
    event = {
        "timestamp": time.strftime("%H:%M:%S"),
        "tool": tool_name,
        "model": usage.model,
        "route": usage.route,
        "tokens": total,
        "latency_ms": int(usage.latency_ms),
    }

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@sheepdestroyer: Sure, kicking off a fresh review of the PR now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • There are unaddressed Reviews & Comments on this PR#
    Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate.
    Make sure you did not miss any reviews or comments.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve all conflicts before continuing.

2 similar comments
@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve all conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Merge master changes & Resolve all conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved. Checkout latest master changes & Resolve all conflicts before continuing.

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • This branch has conflicts that must be resolved.
    Checkout latest master changes & Resolve all conflicts before continuing.

google-labs-jules Bot and others added 8 commits July 1, 2026 14:59
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
google-labs-jules Bot and others added 3 commits July 1, 2026 14:59
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
@sheepdestroyer
sheepdestroyer force-pushed the refactor-record-tool-usage-17779903941173107649 branch from f55b322 to 150f447 Compare July 1, 2026 12:59
@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@gemini review
@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@sheepdestroyer: Sure, kicking off a fresh review of the PR now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the record_tool_usage function in router/main.py to accept a structured ToolUsageRecord dataclass instead of multiple individual parameters, updating all callers and unit tests accordingly. The review feedback suggests improving this refactoring by making the ToolUsageRecord dataclass immutable (frozen=True) and refactoring record_tool_usage to avoid mutating the dataclass fields directly while adding a defensive None check.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread router/main.py
Comment on lines +1064 to +1071
@dataclass
class ToolUsageRecord:
tool_name: str
prompt_tokens: int
completion_tokens: int
model: str
latency_ms: float
route: str = "litellm_fallback"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Making the ToolUsageRecord dataclass immutable by setting frozen=True is a best practice for metric/event records. It prevents accidental modifications after creation, ensures the objects are hashable, and makes the code more robust and easier to reason about.

Suggested change
@dataclass
class ToolUsageRecord:
tool_name: str
prompt_tokens: int
completion_tokens: int
model: str
latency_ms: float
route: str = "litellm_fallback"
@dataclass(frozen=True)
class ToolUsageRecord:
tool_name: str
prompt_tokens: int
completion_tokens: int
model: str
latency_ms: float
route: str = "litellm_fallback"

Comment thread router/main.py
Comment on lines +1074 to 1104
def record_tool_usage(usage: ToolUsageRecord):
"""Accumulates token counts in memory for active tools and tracks request timelines.

File writes are offloaded to a thread pool executor to avoid blocking the
event loop. The 2-second throttle is checked synchronously before
dispatching.
"""
if tool_name == "none":
tool_name = "other"
if usage.tool_name == "none":
usage.tool_name = "other"

total = prompt_tokens + completion_tokens
stats["tool_tokens"][tool_name] = stats["tool_tokens"].get(tool_name, 0) + total
total = usage.prompt_tokens + usage.completion_tokens
stats["tool_tokens"][usage.tool_name] = stats["tool_tokens"].get(usage.tool_name, 0) + total

# Save global prompt/completion metrics
stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + prompt_tokens
stats["completion_tokens"] = stats.get("completion_tokens", 0) + completion_tokens
stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + usage.prompt_tokens
stats["completion_tokens"] = stats.get("completion_tokens", 0) + usage.completion_tokens

# Track routing path distribution
if "routing_paths" not in stats:
stats["routing_paths"] = {"google_oauth_direct": 0, "litellm_fallback": 0}
stats["routing_paths"][route] = stats["routing_paths"].get(route, 0) + 1
stats["routing_paths"][usage.route] = stats["routing_paths"].get(usage.route, 0) + 1

# Append to timeline event stack (in-memory ring buffer + persistent disk backup)
event = {
"timestamp": time.strftime("%H:%M:%S"),
"tool": tool_name,
"model": model,
"route": route,
"tool": usage.tool_name,
"model": usage.model,
"route": usage.route,
"tokens": total,
"latency_ms": int(latency_ms)
"latency_ms": int(usage.latency_ms),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To support defensive programming and prevent runtime errors, we should add a guard check to ensure usage is not None. Additionally, since we should treat ToolUsageRecord as an immutable record (and to support frozen=True), we should avoid mutating usage.tool_name directly. Instead, use a local variable tool_name for the mapping.

def record_tool_usage(usage: ToolUsageRecord):
    """Accumulates token counts in memory for active tools and tracks request timelines.

    File writes are offloaded to a thread pool executor to avoid blocking the
    event loop. The 2-second throttle is checked synchronously before
    dispatching.
    """
    if usage is None:
        return

    tool_name = "other" if usage.tool_name == "none" else usage.tool_name

    total = usage.prompt_tokens + usage.completion_tokens
    stats["tool_tokens"][tool_name] = stats["tool_tokens"].get(tool_name, 0) + total

    # Save global prompt/completion metrics
    stats["prompt_tokens"] = stats.get("prompt_tokens", 0) + usage.prompt_tokens
    stats["completion_tokens"] = stats.get("completion_tokens", 0) + usage.completion_tokens

    # Track routing path distribution
    if "routing_paths" not in stats:
        stats["routing_paths"] = {"google_oauth_direct": 0, "litellm_fallback": 0}
    stats["routing_paths"][usage.route] = stats["routing_paths"].get(usage.route, 0) + 1

    # Append to timeline event stack (in-memory ring buffer + persistent disk backup)
    event = {
        "timestamp": time.strftime("%H:%M:%S"),
        "tool": tool_name,
        "model": usage.model,
        "route": usage.route,
        "tokens": total,
        "latency_ms": int(usage.latency_ms),
    }

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
router/main.py (1)

1732-1739: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Call sites still use positional args for ToolUsageRecord.

The refactor's goal was improved readability/type safety, but all 4 constructors (here, and at lines 1768-1775, 1996-2003, 2052-2059) rely on positional argument order rather than keyword args. This reintroduces the same "guess the argument order" risk the dataclass was meant to eliminate — a future field reorder in ToolUsageRecord would silently break these call sites without a type error. Some tests already use keyword args (e.g. test_record_tool_usage_basic), which would be the safer pattern here too.

♻️ Proposed fix (apply pattern to all 4 call sites)
-                                record_tool_usage(ToolUsageRecord(
-                                    active_tool,
-                                    approx_prompt_tokens,
-                                    token_count,
-                                    model_name,
-                                    latency_ms,
-                                    route="google_oauth_direct",
-                                ))
+                                record_tool_usage(ToolUsageRecord(
+                                    tool_name=active_tool,
+                                    prompt_tokens=approx_prompt_tokens,
+                                    completion_tokens=token_count,
+                                    model=model_name,
+                                    latency_ms=latency_ms,
+                                    route="google_oauth_direct",
+                                ))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@router/main.py` around lines 1732 - 1739, The `ToolUsageRecord` constructor
calls in the `record_tool_usage` call sites still use positional arguments,
which defeats the readability/type-safety refactor. Update all four constructor
usages in the affected route-handling code to pass named keyword arguments
matching `ToolUsageRecord` fields, using the same pattern already used in
`test_record_tool_usage_basic`, so the call sites remain safe if field order
changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@router/main.py`:
- Around line 1732-1739: The `ToolUsageRecord` constructor calls in the
`record_tool_usage` call sites still use positional arguments, which defeats the
readability/type-safety refactor. Update all four constructor usages in the
affected route-handling code to pass named keyword arguments matching
`ToolUsageRecord` fields, using the same pattern already used in
`test_record_tool_usage_basic`, so the call sites remain safe if field order
changes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 54415da2-1d7d-4f3a-89b7-384e28e4190c

📥 Commits

Reviewing files that changed from the base of the PR and between c465873 and 150f447.

📒 Files selected for processing (2)
  • router/main.py
  • test_record_tool_usage.py

@sheepdestroyer

Copy link
Copy Markdown
Owner Author

@jules

  • There are unaddressed Reviews & Comments on this PR#
    Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate.
    Make sure you did not miss any reviews or comments.

Co-authored-by: sheepdestroyer <1377479+sheepdestroyer@users.noreply.github.com>
@sheepdestroyer
sheepdestroyer merged commit e3af6a0 into master Jul 1, 2026
6 checks passed
@sheepdestroyer
sheepdestroyer deleted the refactor-record-tool-usage-17779903941173107649 branch July 1, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant