Skip to content

[Bedrock] Add strict structured-output fallback cache #307

Description

@edelauna

Which API Provider are you using?

AWS Bedrock

Which Model are you using?

Bedrock models that reject strict structured output

What happened?

Bedrock should fail soft when a model rejects strict structured output by retrying without strict mode and caching that unsupported state for a bounded TTL.

Steps to reproduce

  1. Review the structured-output fallback behavior proposed in [ENHANCEMENT] Improve Amazon Bedrock support #125.
  2. Add provider-specific rejection detection and retry behavior.
  3. Cache unsupported strict-output state by Bedrock model or target.

Relevant API REQUEST output

N/A

Additional context

Parent issue: #124
Source PR: #125
Story point estimate: 1

Suggested PR group: PR 2 - runtime

Acceptance criteria:

  • First strict-output rejection retries without strict mode.
  • Subsequent requests for the same cached model/target skip strict mode while the cache entry is valid.
  • Cache expiry restores the normal probe/retry behavior.
  • Tests cover rejection, retry, cache hit, and cache expiry.

The current PR proposes a 30-day cache TTL; call out any desired TTL change during review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions