Fix composite model export (Qwen3 and seq2seq models)#1037
Conversation
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Reviewed the composite-export fan-out and the external-data cleanup. The refactor is clean overall — extracting _run_component_export and the shared resolve_composite_components reads well, and _cleanup_stale_external_data correctly keys off the final on-disk references, so it's safe whether the consolidated re-save ends up inline or external.
A few things worth considering (see inline):
- For composites,
--outputis silently treated as a directory, which is inconsistent with thewinml configfan-out convention and with this PR's own description. - The broad
except Exceptionaround composite detection swallows the intentional loudRuntimeErrorfrom_composite_registry(). - The full model is reloaded once per sub-component.
Nits (no inline):
- The PR description says a
clear_sidecar()method and aclear_sidecarexport were added, but the code actually addsHTPExporter._cleanup_stale_external_data()and exportsget_external_data_files. Please sync the description. - The composite
--input-specsUsageErroris raised after theStarting HTP export...banner prints; hoisting the check above the banner would read cleaner. - The new composite tests mock
load_hf_model/resolve_export_config/export_onnx, so they verify the fan-out plumbing (paths, call counts) but not that the sub-models actually differ — worth a follow-up integration check.
(Note: I couldn't run the suite locally — this machine is win_arm64 and torch ships no matching wheel, so this is a static review.)
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
I left a few inline comments around composite export error handling, output preflight, and test coverage.
…, add cleanup + tests
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
I left one follow-up inline comment on the latest update.
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Approved with the follow-up cleanup comment left as non-blocking.
|
Failure log will be like |
Summary
Refactors
winml exportto support composite (seq2seq) models such as Qwen3, T5, and BART. Previously, runningwinml exporton a composite model would fail because the command treated every model as a single-component export. Now, composite models are automatically detected and each sub-component (e.g. encoder / decoder / decoder-with-past) is exported separately into its own ONNX file.Changes
src/winml/modelkit/loader/resolution.pyresolve_composite_components()— a shared entry point that resolves a composite model's_SUB_MODEL_CONFIG(sub-name → task mapping). Works with both explicit--taskand auto-detected task paths.src/winml/modelkit/commands/export.py_run_component_export()inner function (I/O tensor resolution, config assembly, model load, and ONNX export).<output_stem>_<sub_name>.onnx(matching the siblingwinml configfan-out and the runtime's*_model.onnxdiscovery).RuntimeError/ model-task incompatibilityValueError) are re-raised (the latter as aUsageError, mirroringwinml config) instead of being masked as "not composite".src/winml/modelkit/commands/config.pyresolve_composite_components()helper, removing duplicated resolution logic.src/winml/modelkit/export/htp/exporter.pyclear_sidecar()method toHTPExporterfor cleaning up external data sidecar files after export.src/winml/modelkit/onnx/__init__.pyclear_sidecarsymbol.Tests
--input-specsrejection, that resolution errors surface instead of being swallowed, andresolve_composite_components.