Skip to content

fix(tools): convert unsupported artifact MIME types to text in LoadArtifactsTool#1212

Open
1wos wants to merge 1 commit into
google:mainfrom
1wos:codex/adk-java-load-artifacts-mime-fallback
Open

fix(tools): convert unsupported artifact MIME types to text in LoadArtifactsTool#1212
1wos wants to merge 1 commit into
google:mainfrom
1wos:codex/adk-java-load-artifacts-mime-fallback

Conversation

@1wos
Copy link
Copy Markdown

@1wos 1wos commented May 20, 2026

Link to Issue or Description of Change

Problem:

LoadArtifactsTool appends loaded artifact parts directly into the next model request. When the inline MIME is not accepted by Gemini, the follow-up request fails with Unsupported MIME Type instead of continuing with usable context. Same class of issue as adk-python #4028 (fixed by commit fdc98d5c).

Solution:

Before appending a loaded artifact, classify its MIME type and fall back to a model-safe part:

  • keep inline media supported by Gemini unchanged: image/*, audio/*, video/*, application/pdf
  • convert text-like inline data to text: text/*, application/csv, application/json, application/xml
  • replace unsupported binary inline data with a short placeholder that includes the artifact name, MIME type, and size

MIME normalization strips the ;charset=... parameter, so application/csv; charset=utf-8 is handled the same as application/csv.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added five cases covering the fallback paths:

  • processLlmRequest_unsupportedTextLikeMime_convertsToText
  • processLlmRequest_supportedMime_keepsInlineData
  • processLlmRequest_unsupportedBinaryMime_convertsToPlaceholder
  • processLlmRequest_unsupportedMimeWithoutInlineData_convertsToNoDataPlaceholder
  • processLlmRequest_emptyMime_defaultsToOctetStream
$ ./mvnw -pl core test -Dtest=LoadArtifactsToolTest -DfailIfNoTests=false
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

12/12 pass.

Manual End-to-End (E2E) Tests:

Adapted contrib/samples/helloworld locally (not part of this PR) as a temporary driver: saved a weather.csv artifact with application/csv inline MIME and asked the agent to describe it against the live Gemini API. The agent loaded the artifact, the fallback converted the inline csv to a text part, and the model produced an accurate description:

Saved weather.csv as version 0 with MIME application/csv.
=== E2E: LoadArtifactsTool with application/csv (unsupported inline MIME) ===
Function Call: load_artifacts(artifact_names=[weather.csv])
Function Response: load_artifacts -> {artifact_names=[weather.csv]}
The weather.csv artifact is a CSV file containing weather data. It has two
columns: "city" and "temperature". The file contains temperature readings for
three cities: Seoul (12 degrees), Tokyo (18 degrees), and Singapore (30 degrees).
=== Done ===

Without the fallback, this request fails with Unsupported MIME Type before the model can respond.

Checklist

  • I have read the CONTRIBUTING.md document.
  • My pull request contains a single commit.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Ports the behavior introduced in adk-python by commit fdc98d5c (Close #4028). Follows the "Alignment with adk-python" section of CONTRIBUTING.md.

@hemasekhar-p hemasekhar-p self-assigned this May 21, 2026
@hemasekhar-p
Copy link
Copy Markdown
Contributor

Hi @1wos, Thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you.

@hemasekhar-p
Copy link
Copy Markdown
Contributor

@anFatum, can you please review this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LoadArtifactsTool should handle unsupported artifact MIME types gracefully

2 participants