Fix genai --compile fallback to use relative mirrored ONNX path#1051
Merged
Conversation
When a pipeline stage's offline EPContext compilation failed (or the
stage was already a pre-compiled EPContext model), _prepare_compiled_bundle
patched the stage's genai_config filename to an absolute source path and
skipped mirroring that ONNX into compiled_dir. ort-genai resolves each
stage filename relative to the directory it loads genai_config.json from
(compiled_dir) by naive join, so an absolute Windows path became a broken
"_compiled\C:\...\ctx.onnx" and the bundle failed to load.
Fall back to the original ONNX by its bundle-relative filename and let
_mirror_non_onnx_files link it (plus its weights sidecar) into compiled_dir,
matching the fresh-compile path. The mirror skip-set now holds only stages
actually replaced by a {stage}_ctx.onnx artifact.
Adds regression tests for the compile-failure fallback and the
already-compiled-stage cases.
xieofxie
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Running
winml perf -m <bundle> --runtime winml-genai --device npu --compilefailed to load the bundle whenever a pipeline stage's offline EPContext compilation crashed and fell back to JIT. This was hit in practice benchmarking a qwen3 export on a Snapdragon X Elite NPU: the context stage's offline compile aborted, and the fallback then produced a malformed path and errored out with "Failed to load genai bundle".Root cause
ort-genai resolves each pipeline stage's
filenamerelative to the directory it loadsgenai_config.jsonfrom (the_compileddir) via a naive join, and does not accept absolute Windows paths. In_prepare_compiled_bundle, the compile-failure fallback (and the already-precompiled-stage branch) patched the stagefilenameto an absolute source path and skipped mirroring that ONNX into the compiled dir. ort-genai then joined the compiled dir onto the absolute path, yielding a broken_compiled\C:\...\ctx.onnxthat did not exist.Fix
Reference the original ONNX by its bundle-relative filename and let
_mirror_non_onnx_filessymlink it (plus its external-weights.datasidecar) into the compiled dir, exactly like the fresh-compile path already does. The mirror skip-set now contains only the stages that were actually replaced by a{stage}_ctx.onnxartifact, so fallback / already-compiled stages get mirrored in correctly.Notes for reviewers
0xC0000005in the EPContext compiler subprocess) is a separate pre-existing issue and is not addressed here; the fix ensures the JIT fallback after that crash loads correctly rather than dying on a bad path..datasidecar) is physically present in the compiled dir.--compilenow completes (context stage falls back to JIT, bundle loads, benchmark runs) with the malformed-path error gone.