emrg: read tool returns vision-format image blocks for local images (rant 2026-08-24T14:36:01) - #955
Conversation
…rant 2026-08-24T14:36:01)
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle 144335+1. Reviewed against rant 2026-08-24T14:36:01:
- read_tool.py returns a structured image ref ({"type":"image",path,mime}) for .png/.jpg/.jpeg/.gif/.webp instead of erroring on binary read; 256KB cap with a concrete resize hint; extension matching is case-insensitive.
- daemon._tool_content_for_llm() correctly degrades: vision model → OpenAI [text, image_url] data-URL block; non-vision model → text placeholder; non-image results passthrough unchanged; missing file → graceful '[Image unavailable]' warning instead of a crash.
- All 3 tool-message construction sites (main loop + 2 resume paths) are wired through the helper.
- Tests cover vision/non-vision/passthrough/missing-image + read_tool mime/size/uppercase/description; Agent.md count synced 1049. CI green (test 1m21s + test-windows 59s, run 32698965107).
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM (2/3) — cycle 2026-08-24T15:03, independent re-verification. Head cd2b995 unchanged, MERGEABLE/CLEAN, CI double-green (test 1m21s + test-windows 59s, run 32698965107). Change set matches the rant 2026-08-24T14:36:01 acceptance items (vision-format image refs from ReadTool; daemon-side conversion with non-vision degradation; 3 tool-message sites wired; 9 new tests; Agent.md count synced).
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM (3/3) — cycle 2026-08-24T15:10, final independent verification. Head cd2b995 unchanged, MERGEABLE/CLEAN, CI double-green (test 1m21s + test-windows 59s, run 32698965107). Three consecutive approvals from three distinct cycles (145700, 150307, 151015), no requested changes in between. Change set verified against rant 2026-08-24T14:36:01: ReadTool returns vision-format image refs for local images, daemon._tool_content_for_llm converts to OpenAI vision content blocks with non-vision degradation, all 3 tool-message sites wired, 9 new tests, Agent.md count synced. Merging.
Summary
Rant 2026-08-24T14:36:01: users with a vision-capable model (e.g. deepseek-v4-flash-vision-exp) had no way to let the model see a local image —
readon a binary file returnedUnicodeDecodeError→ "binary file?" error, and bash base64 output is text, not a vision image block. Only clipboard paste (TUI/image) could reach vision format.Changes
{"type": "image", "path": ..., "mime": ...}instead of a binary error;MAX_IMAGE_SIZE(256KB) cap with a resize hint (sips/ImageMagick) for oversized images; tool description updated to advertise image support._tool_content_for_llm()helper wired into all 3 tool-message sites (main tool loop + 2 continue paths). It converts the read-tool image ref into an OpenAI vision content block (data-URL base64) for vision-capable models, degrades to a text placeholder for non-vision models, and passes non-image results through unchanged.Acceptance
read /path/to/img.pngreturns a vision block instead of a binary error