Skip to content

fix(lib): only raise finish-reason errors when there is parseable input - #3589

Open
hsusul wants to merge 1 commit into
openai:mainfrom
hsusul:fix/parse-length-guard-non-parse
Open

fix(lib): only raise finish-reason errors when there is parseable input#3589
hsusul wants to merge 1 commit into
openai:mainfrom
hsusul:fix/parse-length-guard-non-parse

Conversation

@hsusul

@hsusul hsusul commented Aug 10, 2026

Copy link
Copy Markdown
  • I understand that this repository is auto-generated and my pull request may not be merged

This change is in hand-maintained code, not generated code. Both files live under
the Stainless-exempt paths (src/openai/lib/… and tests/lib/…); neither carries the
File generated from our OpenAPI spec by Stainless header, so it won't be reverted on
the next generation.

Changes being requested

parse_chat_completion (src/openai/lib/_parsing/_completions.py) raises
LengthFinishReasonError / ContentFilterFinishReasonError for any choice whose
finish_reason is length / content_filter, regardless of whether structured-output
parsing was actually requested.

That is inconsistent with the streaming accumulator
(ChatCompletionStreamState._accumulate_chunk), which only raises these errors when
has_parseable_input(...) is true:

# src/openai/lib/streaming/chat/_completions.py
if has_parseable_input(response_format=self._response_format, input_tools=self._input_tools):
    if choice.finish_reason == "length":
        raise LengthFinishReasonError(completion=completion_snapshot)
    if choice.finish_reason == "content_filter":
        raise ContentFilterFinishReasonError()

Because get_final_completion() funnels through parse_chat_completion, a plain
client.chat.completions.stream(...) — no response_format, no parseable tools — that
stops at the token limit iterates to completion without raising (the accumulator
suppresses it), but then raises LengthFinishReasonError from get_final_completion():

from openai.lib.streaming.chat import ChatCompletionStreamState
from openai.types.chat import ChatCompletionChunk
from openai.types.chat.chat_completion_chunk import Choice, ChoiceDelta

state = ChatCompletionStreamState()  # no response_format / tools
def chunk(delta, finish): return ChatCompletionChunk.construct(
    id="c", object="chat.completion.chunk", created=0, model="gpt-4o",
    choices=[Choice.construct(index=0, delta=delta, finish_reason=finish)])

state.handle_chunk(chunk(ChoiceDelta.construct(role="assistant", content="cut o"), None))
state.handle_chunk(chunk(ChoiceDelta.construct(), "length"))  # no raise here
state.get_final_completion()  # -> openai.LengthFinishReasonError

This is surprising for two reasons:

  1. The equivalent non-streaming client.chat.completions.create() never raises for the
    same response — it just returns a completion with finish_reason="length".
  2. helpers.md documents the length / content-filter raise only under "Differences
    from .create()"
    for chat.completions.parse() (structured outputs); it is not a
    documented behavior of a plain .stream().

Fix

Compute has_parseable_input(...) once in parse_chat_completion and guard both raises
with it, mirroring the accumulator. Behavior is unchanged whenever there is something to
parse:

call before after
.parse(response_format=Model) at length raises raises (unchanged)
.parse(tools=[strict fn]) at length raises raises (unchanged)
.stream(response_format=Model) at length raises (mid-stream) raises (unchanged)
plain .stream()get_final_completion() at length raises returns completion
plain .parse() (no format/tools) at length raises returns completion (now matches .create())

has_parseable_input is already defined in the same module, so no new import is needed.

Tests

tests/lib/chat/test_completions_streaming.py:

  • test_non_parse_stream_terminal_finish_reason_does_not_raise[length|content_filter]
    drives ChatCompletionStreamState (no network) with a content chunk + a terminal
    finish chunk; asserts get_final_completion() returns the accumulated completion with
    the content and finish_reason preserved. Fails on main for both parameters.
  • test_parse_stream_length_finish_still_raises — with a response_format, the terminal
    length chunk still raises LengthFinishReasonError (guards the unchanged path).

Validation

Run in an isolated worktree off upstream/main @ 0c09a3fe (Python 3.9, pydantic 2.12.5,
pyright 1.1.399):

  • pytest tests/lib/chat tests/lib/test_pydantic.py -o addopts="" → all pass; the three
    new tests fail on main for the right reason and pass after the change.
  • pytest tests/lib -o addopts="" → 258 passed. The single failure
    (test_bedrock_auth_conformance.py::test_retry_signing_fixture) is pre-existing and
    network-dependent — it fails identically on unmodified upstream/main and is unrelated
    to this change.
  • ruff check + ruff format --check on both files → clean.
  • pyright on src/openai/lib/_parsing/_completions.py → 0 errors; the test file's
    strict-mode error count is unchanged by this PR.

No API key, paid call, or live service was used.

Additional context & links

`parse_chat_completion` raised `LengthFinishReasonError` /
`ContentFilterFinishReasonError` for any choice whose `finish_reason` was
`length` / `content_filter`, regardless of whether structured-output parsing
was requested. This is inconsistent with the streaming accumulator
(`ChatCompletionStreamState._accumulate_chunk`), which only raises these when
`has_parseable_input` is true.

As a result a plain `client.chat.completions.stream(...)` (no `response_format`
and no parseable tools) that stops at the token limit would iterate to
completion without raising, but then raise `LengthFinishReasonError` from
`get_final_completion()` — even though `client.chat.completions.create()`
never raises for the same response and the length/content-filter restriction
is documented only for `.parse()` (structured outputs).

Guard both raises with `has_parseable_input`, mirroring the accumulator, so
the errors are only raised when there is actually something to parse.
@hsusul
hsusul requested a review from a team as a code owner August 10, 2026 00:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f33b811813

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# mirror the streaming accumulator (`ChatCompletionStreamState`), which guards these
# errors with `has_parseable_input`, and leave the completion untouched — matching
# `chat.completions.create()`.
raise_on_incomplete = has_parseable_input(response_format=response_format, input_tools=input_tools)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve finish errors for one-shot tool iterables

When tools is a generator (which the public Iterable annotation permits), both sync and async chat.completions.parse() first consume it in _validate_input_tools (completions.py:179 and completions.py:1790) and then pass the exhausted iterator here. Consequently input_tools becomes empty and raise_on_incomplete is false, so a request that supplied a strict parseable tool but finishes with length or content_filter now silently returns an incomplete completion instead of raising. Materialize the tools before validation or otherwise preserve their parseability state.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant