Eliminate redundant NCHW↔NHWC permute_copy and NHWC-safe view_copy transposes in ToTosaMemoryFormatPass (#18314) by 3l1 · Pull Request #18314 · pytorch/executorch

3l1 · 2026-03-19T08:46:23Z

Summary:

Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes:

NHWC-safe reshape detection: When a 4D→4D view_copy has monotonic
shape_indices on the raw shapes and preserves both the batch dim (index 0)
and the last dimension (NHWC channel) alone in their output groups, skip
inserting input/output transposes. The view_copy can operate directly on
NHWC data.
Redundant permute_copy elimination: Model-level permute_copy ops whose
permutation matches channels_last_order (NCHW→NHWC) or its inverse
(NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant
with the tosa_dim_order annotation. Replace them with view_copy (identity
reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute
models (NCHW input from placeholder) are not affected.

bypass-github-export-checks
bypass-github-pytorch-ci-checks
bypass-github-executorch-ci-checks

Reviewed By: digantdesai

Differential Revision: D97266678

pytorch-bot · 2026-03-19T08:46:29Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18314

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

meta-codesync · 2026-03-19T08:46:33Z

@3l1 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D97266678.

github-actions · 2026-03-19T08:47:12Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

AdrianLundell · 2026-03-19T13:08:28Z

Hi, thanks for the PR! This is a complex topic to get right in all cases and FYI we are also planning on improving this internally so it is very nice to get some help with that. I see there are some errors in our unittests so looks like there are a few edge-cases to iron out before a proper review. Let us know if you have any questions about the current logic to help with this.

In the meanwhile that I have two comments:

The first optimization seems like a slighly more narrow condition of the "is_channel_reshape" function, do you think it could fit into there rather than a separate function?
For the second optimization, have you observed performance regressions due to noop transposes? Our working assumption has been that this is optimized away by the compiler.

3l1 · 2026-03-19T17:00:23Z

hey @AdrianLundell thanks for the feedback

yes that is something I can look into wrapping up into the existing function
yes we see on u55 (not u85) issues where regor was not optimizing away all of them, causing perf issues on firmware

(also fixing failing unit tests)

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Differential Revision: D97266678

digantdesai

Review automatically exported from Phabricator review in Meta.

3l1 · 2026-03-25T16:49:03Z

checking Gemma3n tests...

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Reviewed By: digantdesai Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Reviewed By: digantdesai Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Reviewed By: digantdesai Differential Revision: D97266678

…ansposes in ToTosaMemoryFormatPass (#18314) Summary: Pull Request resolved: #18314 Two optimizations in ToTosaMemoryFormatPass to reduce TOSA TRANSPOSE nodes: 1. **NHWC-safe reshape detection:** When a 4D→4D view_copy has monotonic shape_indices on the raw shapes and preserves both the batch dim (index 0) and the last dimension (NHWC channel) alone in their output groups, skip inserting input/output transposes. The view_copy can operate directly on NHWC data. 2. **Redundant permute_copy elimination:** Model-level permute_copy ops whose permutation matches channels_last_order (NCHW→NHWC) or its inverse (NHWC→NCHW) AND whose input already has NHWC tosa_dim_order are redundant with the tosa_dim_order annotation. Replace them with view_copy (identity reshape) to avoid generating TOSA TRANSPOSE nodes. Standalone permute models (NCHW input from placeholder) are not affected. bypass-github-export-checks bypass-github-pytorch-ci-checks bypass-github-executorch-ci-checks Reviewed By: digantdesai Differential Revision: D97266678

3l1 requested a review from digantdesai as a code owner March 19, 2026 08:46

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 19, 2026

meta-codesync bot added fb-exported meta-exported labels Mar 19, 2026

meta-codesync bot changed the title ~~Eliminate redundant NCHW↔NHWC permute_copy and NHWC-safe view_copy transposes in ToTosaMemoryFormatPass (#18167)~~ Eliminate redundant NCHW↔NHWC permute_copy and NHWC-safe view_copy transposes in ToTosaMemoryFormatPass (#18314) Mar 19, 2026

meta-codesync bot force-pushed the export-D97266678 branch from e505b3e to 7ebe074 Compare March 19, 2026 09:00

meta-codesync bot force-pushed the export-D97266678 branch from 7ebe074 to 1e0b323 Compare March 19, 2026 19:55

meta-codesync bot force-pushed the export-D97266678 branch from 1e0b323 to 92fa406 Compare March 19, 2026 19:56

3l1 force-pushed the export-D97266678 branch from 92fa406 to 3be2196 Compare March 19, 2026 19:58

meta-codesync bot force-pushed the export-D97266678 branch from 3be2196 to 8b742ba Compare March 19, 2026 20:08

3l1 force-pushed the export-D97266678 branch from 8b742ba to 4f89292 Compare March 19, 2026 20:12

meta-codesync bot force-pushed the export-D97266678 branch from 4f89292 to ba24f69 Compare March 19, 2026 20:31

meta-codesync bot force-pushed the export-D97266678 branch from ba24f69 to 7b1eff5 Compare March 19, 2026 21:25

meta-codesync bot force-pushed the export-D97266678 branch from 7b1eff5 to e02287c Compare March 19, 2026 21:25

3l1 force-pushed the export-D97266678 branch from e02287c to 58e8ec2 Compare March 19, 2026 21:28

meta-codesync bot force-pushed the export-D97266678 branch from 7722299 to 8c9cd27 Compare March 24, 2026 21:57

meta-codesync bot force-pushed the export-D97266678 branch from 8c9cd27 to d37271b Compare March 24, 2026 21:57

3l1 force-pushed the export-D97266678 branch from d37271b to 38ab0f4 Compare March 24, 2026 22:00

meta-codesync bot force-pushed the export-D97266678 branch from 38ab0f4 to 524f633 Compare March 24, 2026 22:03

3l1 force-pushed the export-D97266678 branch from 524f633 to d9a8827 Compare March 24, 2026 22:06

meta-codesync bot force-pushed the export-D97266678 branch from d9a8827 to c05fece Compare March 25, 2026 01:53

digantdesai approved these changes Mar 25, 2026

View reviewed changes

meta-codesync bot force-pushed the export-D97266678 branch from c05fece to 0ea15d1 Compare March 25, 2026 21:24

meta-codesync bot force-pushed the export-D97266678 branch from 0ea15d1 to 288227c Compare March 25, 2026 21:47

meta-codesync bot force-pushed the export-D97266678 branch from 288227c to 9c5dbf1 Compare March 25, 2026 21:48

3l1 force-pushed the export-D97266678 branch from 9c5dbf1 to 44f225f Compare March 25, 2026 21:50

meta-codesync bot force-pushed the export-D97266678 branch from 44f225f to fda5066 Compare March 25, 2026 21:58

meta-codesync bot force-pushed the export-D97266678 branch from fda5066 to 5e1384b Compare March 25, 2026 22:48

meta-codesync bot force-pushed the export-D97266678 branch from 5e1384b to 751766b Compare March 26, 2026 01:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Eliminate redundant NCHW↔NHWC permute_copy and NHWC-safe view_copy transposes in ToTosaMemoryFormatPass (#18314)#18314

Eliminate redundant NCHW↔NHWC permute_copy and NHWC-safe view_copy transposes in ToTosaMemoryFormatPass (#18314)#18314
3l1 wants to merge 1 commit intomainfrom
export-D97266678

3l1 commented Mar 19, 2026 •

edited by meta-codesync bot

Loading

Uh oh!

pytorch-bot bot commented Mar 19, 2026 •

edited

Loading

Uh oh!

meta-codesync bot commented Mar 19, 2026

Uh oh!

github-actions bot commented Mar 19, 2026

Uh oh!

AdrianLundell commented Mar 19, 2026

Uh oh!

3l1 commented Mar 19, 2026

Uh oh!

digantdesai left a comment

Uh oh!

3l1 commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

3l1 commented Mar 19, 2026 • edited by meta-codesync bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18314

Uh oh!

meta-codesync bot commented Mar 19, 2026

Uh oh!

github-actions bot commented Mar 19, 2026

This PR needs a release notes: label

Uh oh!

AdrianLundell commented Mar 19, 2026

Uh oh!

3l1 commented Mar 19, 2026

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

3l1 commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

3l1 commented Mar 19, 2026 •

edited by meta-codesync bot

Loading

pytorch-bot bot commented Mar 19, 2026 •

edited

Loading

This PR needs a `release notes:` label