Skip to content

fix(perf): per-module benchmark works for ResNet (and similar HF models)#586

Merged
xieofxie merged 4 commits into
mainfrom
hualxie/fix_module
May 13, 2026
Merged

fix(perf): per-module benchmark works for ResNet (and similar HF models)#586
xieofxie merged 4 commits into
mainfrom
hualxie/fix_module

Conversation

@xieofxie

Copy link
Copy Markdown
Contributor

Summary

Fixes winml perf --module <ClassName> on HuggingFace models like microsoft/resnet-50, which previously failed with AttributeError: <Parent> has no attribute '<...>' and, once that was past, forward() missing 1 required positional argument.

Two root causes:

  1. Wrong parent class instantiated. _perf_modules passed the CLI task (typically None) to _instantiate_parent_model. When the model_type's first supported task differs from the architectures-derived task used by generate_hf_build_config, the parent class mismatches the recorded module_path. For microsoft/resnet-50, build configs use ResNetForImageClassification (paths prefixed with resnet.), but perf was instantiating bare ResNetModelAttributeError: ResNetModel has no attribute 'resnet'. Fix: re-resolve the parent task from the model_id via resolve_loader_config, matching the path used to compute module_path.

  2. Wrong input tensor names. _build_submodule_config hardcoded input names as input_0, input_1, …, but PyTorch submodules expect specific positional/keyword names (e.g., ResNetStage.forward(input), ResNetBottleNeckLayer.forward(hidden_state), ResNetEmbeddings.forward(pixel_values)). Calling submodule(input_0=tensor) raises forward() missing 1 required positional argument. Fix: propagate forward-arg names through SubmoduleInfo.input_names (sourced from the existing hook-capture data, with a inspect.signature fallback) and use them when building InputTensorSpec. Falls back to input_{i} only when names can't be discovered.

End-to-end on a CPU box:

$ winml perf -m microsoft/resnet-50 --module ResNetStage --ep cpu --device cpu
                Per-Module Perf: ResNetStage
| Module Path             | Mean (ms) | P90 (ms) | Min  | Max  |
| resnet.encoder.stages.0 |      4.16 |     5.22 | 3.30 | 5.22 |
| resnet.encoder.stages.1 |     14.04 |    24.09 | 8.69 | 24.09 |
| resnet.encoder.stages.2 |      7.41 |     8.65 | 4.93 | 8.65 |
| resnet.encoder.stages.3 |      5.20 |     7.09 | 3.59 | 7.09 |

Closes #192

@xieofxie xieofxie requested a review from a team as a code owner May 12, 2026 02:02
@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

Comment thread src/winml/modelkit/config/build.py
Comment thread src/winml/modelkit/commands/perf.py Outdated
Comment thread src/winml/modelkit/config/build.py Outdated
Comment thread src/winml/modelkit/config/build.py Outdated
Comment thread src/winml/modelkit/commands/perf.py
@xieofxie xieofxie merged commit a201930 into main May 13, 2026
9 checks passed
@xieofxie xieofxie deleted the hualxie/fix_module branch May 13, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wmk perf --module: module path construction fails for ResNet (AttributeError: ResNetModel has no attribute 'resnet')

3 participants