[Fix] Images returned by MCP tools are not displayed in chat#1025
[Fix] Images returned by MCP tools are not displayed in chat#1025zoomote[bot] wants to merge 2 commits into
Conversation
…d resource images to the model
|
No new code issues found. The new commit is test-only: the data-URL blob test matches the Reviewed 3ce2f3b |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@zoomote fix the code coverage |
|
I'm on it. I routed this request into the existing PR task so follow-up work stays on one Roomote thread for this PR, and I'll keep updates here. See task |
|
Coverage gaps addressed in
These target the exact lines Codecov flagged (1 missing + 1 partial in |
Related GitHub Issue
Closes: #1017
Description
When an MCP tool returned image content, Zoo Code only showed the
[N image(s) received]placeholder text in chat — the actual image data was silently dropped from the UI even though it was persisted on themcp_server_responsemessage.Two root causes, both fixed:
mcp_server_responsesay message (message.images), butChatRow.tsxhad nocase "mcp_server_response"— it fell through to the default case, which renders only markdown text. Added an explicit case that renders the response text plus each attached image via the existingImageBlockcomponent (the same pattern used for assistanttextmessages with images). The default say case now also rendersmessage.imageswhen present, so no future say type can silently drop attached images.resourceimage blobs were discarded before reaching the model.processToolContentinUseMcpToolTool.tsdestructuredblobaway fromresourcecontent blocks. Image resources (mimeTypestarting withimage) are now extracted into data URLs, mirroring the existing behavior ofaccess_mcp_resource, so they are both displayed in chat and forwarded to the model as image blocks. Non-image blobs are still serialized as JSON text as before.MCP
type: "image"content blocks were already wired through to the model correctly (subject to the model's declaredsupportsImagescapability); this PR fixes the display gap and the resource-blob gap.Test Procedure
src/core/tools/__tests__/useMcpToolTool.spec.ts: an MCP tool result with an embedded imageresourceblob is extracted into a data URL and passed tosay/pushToolResult; a data-URL-formatted blob is not double-prefixed; a non-image (text/plain) resource blob is not extracted.webview-ui/src/components/chat/__tests__/ChatRow.mcp-server-response.spec.tsx:mcp_server_responsemessages render one image, multiple images, and no image correctly, and say types using the default renderer also display attached images.pnpm testfrom the repo root: passed (425 test files, 7010 tests).pnpm lintandpnpm check-types(turbo, all 14 packages): passed.game_screenshot), invoking it in chat with an image-capable model, and confirming the image renders inline in the chat response.Pre-Submission Checklist
Screenshots / Videos
Not captured: this change is inside the VS Code extension webview, which has no browser-reachable surface in the automated environment. Rendering is covered by the new
ChatRowcomponent tests, which assert the images returned by MCP tools are displayed in the chat row.Documentation Updates
The existing docs for
use_mcp_toolandaccess_mcp_resourcealready describe image content being processed and displayed; this fix restores that documented behavior.Additional Notes
Image blocks are only sent to the model when the active model profile declares image support (
supportsImages); that capability gating is pre-existing and unchanged.Get in Touch
Available via this PR's comments for any reviewer questions.
Linked work items
Closes #1017