Skip to content

Fix genai --compile fallback to use relative mirrored ONNX path#1051

Merged
DingmaomaoBJTU merged 1 commit into
mainfrom
dingmaomaobjtu-qwen3-export-perf
Jul 6, 2026
Merged

Fix genai --compile fallback to use relative mirrored ONNX path#1051
DingmaomaoBJTU merged 1 commit into
mainfrom
dingmaomaobjtu-qwen3-export-perf

Conversation

@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator

Why

Running winml perf -m <bundle> --runtime winml-genai --device npu --compile failed to load the bundle whenever a pipeline stage's offline EPContext compilation crashed and fell back to JIT. This was hit in practice benchmarking a qwen3 export on a Snapdragon X Elite NPU: the context stage's offline compile aborted, and the fallback then produced a malformed path and errored out with "Failed to load genai bundle".

Root cause

ort-genai resolves each pipeline stage's filename relative to the directory it loads genai_config.json from (the _compiled dir) via a naive join, and does not accept absolute Windows paths. In _prepare_compiled_bundle, the compile-failure fallback (and the already-precompiled-stage branch) patched the stage filename to an absolute source path and skipped mirroring that ONNX into the compiled dir. ort-genai then joined the compiled dir onto the absolute path, yielding a broken _compiled\C:\...\ctx.onnx that did not exist.

Fix

Reference the original ONNX by its bundle-relative filename and let _mirror_non_onnx_files symlink it (plus its external-weights .data sidecar) into the compiled dir, exactly like the fresh-compile path already does. The mirror skip-set now contains only the stages that were actually replaced by a {stage}_ctx.onnx artifact, so fallback / already-compiled stages get mirrored in correctly.

Notes for reviewers

  • This only fixes the fallback behavior. The underlying QNN offline-compile crash (native 0xC0000005 in the EPContext compiler subprocess) is a separate pre-existing issue and is not addressed here; the fix ensures the JIT fallback after that crash loads correctly rather than dying on a bad path.
  • Added two regression tests covering the compile-failure fallback and the already-precompiled-stage case, asserting the stage is referenced by a relative filename and that the ONNX (+ .data sidecar) is physically present in the compiled dir.
  • Verified end-to-end on the NPU: --compile now completes (context stage falls back to JIT, bundle loads, benchmark runs) with the malformed-path error gone.

When a pipeline stage's offline EPContext compilation failed (or the
stage was already a pre-compiled EPContext model), _prepare_compiled_bundle
patched the stage's genai_config filename to an absolute source path and
skipped mirroring that ONNX into compiled_dir. ort-genai resolves each
stage filename relative to the directory it loads genai_config.json from
(compiled_dir) by naive join, so an absolute Windows path became a broken
"_compiled\C:\...\ctx.onnx" and the bundle failed to load.

Fall back to the original ONNX by its bundle-relative filename and let
_mirror_non_onnx_files link it (plus its weights sidecar) into compiled_dir,
matching the fresh-compile path. The mirror skip-set now holds only stages
actually replaced by a {stage}_ctx.onnx artifact.

Adds regression tests for the compile-failure fallback and the
already-compiled-stage cases.
@DingmaomaoBJTU DingmaomaoBJTU requested a review from a team as a code owner July 6, 2026 02:30
@DingmaomaoBJTU DingmaomaoBJTU merged commit 8c88d42 into main Jul 6, 2026
8 checks passed
@DingmaomaoBJTU DingmaomaoBJTU deleted the dingmaomaobjtu-qwen3-export-perf branch July 6, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants