Agent Mode: Streamed tool_calls from vLLM ignored - no action performed

[halom651]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:macOS
- Continue version: continue dev
- IDE version: pycharm/ VScode
- Model: qwen3-32b with vLLM v0.8.5


### Description

Hi Continue team,
We're running qwen3-32b with vLLM v0.8.5 using the OpenAI-compatible API, and testing Agent Mode with tool_calls enabled.
The model returns a valid tool_calls response via streamed SSE, but Continue does not display any output or perform any action. There are no errors- the output is silently ignored.

Sample streamed response from vLLM:

data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"name":"builtin_create_new_file"}}]}}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"arguments":"{"filepath": ""}}]}}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"arguments":"file"}}]}}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"arguments":".txt"}}]}}]}
...
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"arguments":" , \"contents\": \"test\""}}]}}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"tool_calls":[{"function":{"arguments":"\"}"}}]}}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"content":""},"finish_reason":"tool_calls"}]}

This follows OpenAI's (vLLM) streaming format for tool calling, but it seems Continue doesn’t currently reconstruct or act on these chunks.

Launch flags used for vLLM:

python3 -m vllm.entrypoints.openai.api_server \
--served-model-name qwen3-32b \
--quantization bitsandbytes \
--dtype float16 \
--gpu-memory-utilization 0.9 \
--trust-remote-code \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--chat-template examples/tool_chat_template_hermes.jinja \
--max-model-len 32768

It looks like the model is emitting a valid tool_calls stream, but Continue may not fully support:
streamed function/tool calling
or the hermes parser format
or partial assembly of arguments across chunks
Could you confirm whether this format is currently supported, or if there are specific constraints on the tool call structure for Continue to handle it?
Happy to share full logs or config if needed

### To reproduce

_No response_

### Log output

```Shell
date: Sun, 04 May 2025 13:03:12 GMT
server: uvicorn
content-type: text/event-stream; charset=utf-8
connection: close
transfer-encoding: chunked

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"content":"<think>"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"content":"\n\n"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"content":"</think>"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"content":"\n\n"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-3c3d359d32b34573a6ff324ca0951c42","type":"function","index":0,"function":{"name":"builtin_create_new_file"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"filepath\": \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"file"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":".txt"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\","}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"contents"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":" \"\""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-c6581b7a2d4b4a108e90c20c3e5c2282","object":"chat.completion.chunk","created":1746363793,"model":"agent","choices":[{"index":0,"delta":{"content":""},"logprobs":null,"finish_reason":"tool_calls","stop_reason":null}]}

[sestinj]

It sounds like there might be an issue with the Hermes parser format given this other related issue: #5469

[Fly-Pluche]

Please tell me this project's support parser type of --tool-call-parser.
@sestinj

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[sfeese]

I'm observing the same with continue v1.1.75 and openai gpt4o. Tool calling doesn't work with streaming responses from openai, the function arguments are not correctly parsed from the stream.

[YFrendo]

Same with gpt-oss120B on VLLM (0.11.0).
No action performed

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.