Skip to content

NXP backend: Enable MM and AddMM with new Neutron flow.#20278

Open
MartinPavella wants to merge 6 commits into
pytorch:mainfrom
nxp-upstream:nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow
Open

NXP backend: Enable MM and AddMM with new Neutron flow.#20278
MartinPavella wants to merge 6 commits into
pytorch:mainfrom
nxp-upstream:nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow

Conversation

@MartinPavella

@MartinPavella MartinPavella commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR enables the MM and AddMM operators in the NXP backend using the new Neutron C MLIR flow.

Test plan

Unit tests provided.

cc @robert-kalmar @JakeStevens @digantdesai @rascani

@MartinPavella MartinPavella self-assigned this Jun 15, 2026
@MartinPavella MartinPavella added module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate labels Jun 15, 2026
@pytorch-bot

pytorch-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20278

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit b10f3f9 with merge base 6021a58 (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 15, 2026

Copy link
Copy Markdown

CLA Not Signed

@MartinPavella

Copy link
Copy Markdown
Collaborator Author

@novak-vaclav, @irtrukhina please feel free to have a look as well.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 15, 2026
@MartinPavella MartinPavella force-pushed the nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow branch from 9c886de to 3495635 Compare June 17, 2026 06:36
@MartinPavella MartinPavella force-pushed the nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow branch from 3495635 to 605f986 Compare June 17, 2026 13:54
@MartinPavella MartinPavella force-pushed the nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow branch 2 times, most recently from 3044d1d to 896a8c2 Compare June 19, 2026 07:50
Comment thread backends/nxp/backend/ir/converter/node_converter.py Outdated

@roman-janik-nxp roman-janik-nxp left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, need to wait on internal tests.

@MartinPavella MartinPavella force-pushed the nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow branch from 896a8c2 to e3aa36b Compare June 22, 2026 07:51
@roman-janik-nxp

Copy link
Copy Markdown
Collaborator

@roman-janik-nxp roman-janik-nxp left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@novak-vaclav novak-vaclav left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just a few comments and questions, otherwise very good 👌



# The edge operator signature is: aten.addmm(bias, input, weight, *, beta=1, alpha=1)
MAIN_INPUT_IDX = 1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicely explained!

def convert(self, node: Node):
"""Convert the `aten.addmm` operator to NeutronIR `FullyConnected`.
The schema is:
addmm(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is the schema of high level torch API, however in aten, the addmm operator's signature looks different.
I find this a bit confusing, since the node_converter deals with only aten operators.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? What lead you to believe the schema should be different?
When I debug the lowering and stop in the AddMMConverter, I can see the schema of the exact operator that is being converted, and it is exactly equal to what's in the doc-string (see attached).

Image

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I had in mind was I'd rather see directly the "schema" of node.args, since that is what we directly work with in node_converter. Something like node.args = [input_node, mat1, ...].
I thought the schema was taken from the documentation online, which might be less reliable than directly identifing the nodes in node.args. However if it is taken from the node attribute directly, then it's fine by me.

Comment thread backends/nxp/tests/generic_tests/test_integration.py

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just thinking, we could make a precommit hook that would check that for each modified file in a commit, if there is an NXP license, the year must be up to date.
Something like: get modified files -> for each file, read the first line -> if the line starts with "# Copyright", it must also contain "2026"
What do you think? I don't know if we can just create precommit hooks in an opensource repository. An alternative might be a special CI test in .github/workflows?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea!
Just a note, only "significant" changes to a file require the license to be updated. The word "significant" is vague and up to our interpretation (for example for the changes to test_integration.py here, I deemed the changes insignificant, as I just added a comment, updates some indices, and replaced operators with their aliases. No functional changes were made. But someone else may consider this significant anyway.).

Regarding the automatic check, please feel free to bring it up during the architecture meeting on Monday. I quite like it :)

Comment thread backends/nxp/tests/use_qat.py
@MartinPavella MartinPavella force-pushed the nxg01483/EIEX-862-add-mm-addmm-support-using-new-neutron-flow branch from e3aa36b to b10f3f9 Compare June 26, 2026 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants