Improve ELU support#19694
Open
AdrianLundell wants to merge 3 commits into
Open
Conversation
- Adds support for different scale and input_scale values - Adds support for related SELU and CELU operators, corresponding to ELU with particular scales. - Use initial float values for alpha, scale and input_scale rather than rounded values. Signed-off-by: Adrian Lundell <adrian.lundell@arm.com> Change-Id: I34dc97ce661213ffdcc9cf122028a115211b56e2
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com> Change-Id: I541b65ce989d2dff5a0af3f8605e946b1055a899
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19694
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 2 New Failures, 4 Unrelated FailuresAs of commit cc9d7e2 with merge base 7724fd7 ( NEW FAILURES - The following jobs have failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
BROKEN TRUNK - The following jobs failed but was present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
AdrianLundell
added
the
partner: arm
github-actions
Bot
added
ciflow/trunk
module: arm
AdrianLundell
added
release notes: none
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Arm backend handling of the ELU-family activations by adding explicit support for SELU and CELU through conversion/decomposition passes, updating operator support/partitioning metadata, and expanding the ELU test suite accordingly.
Changes:
- Add pass to convert
SELU/CELUto parameterizedELU, and update ELU decomposition to respectalpha/scale/input_scale. - Extend quantization annotation and TOSA operator support lists to include
selu/celu, and adjust partitioner “do-not-decompose” policy for float. - Refactor/expand ELU tests to cover ELU parameters as well as
nn.SELUandnn.CELU, and update related docs.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| backends/arm/tosa/partitioner.py | Prevents decomposition of selu/celu in FP flows so backend passes can handle them. |
| backends/arm/test/ops/test_elu.py | Refactors test structure and adds SELU/CELU + additional ELU parameter coverage across pipelines. |
| backends/arm/scripts/docgen/vgf/vgf-getting-started-tutorial.md.in | Clarifies compiler script positioning (quick tests vs production API). |
| backends/arm/scripts/docgen/ethos-u/ethos-u-getting-started-tutorial.md.in | Same documentation clarification as VGF tutorial. |
| backends/arm/quantizer/quantization_annotator.py | Adds selu/celu to one-to-one quantization annotation set. |
| backends/arm/operator_support/tosa_profile_supported_op_lists.py | Declares selu/celu as supported ops in relevant TOSA profiles. |
| backends/arm/_passes/insert_table_ops.py | Updates ELU table-op computation to use stored float params; adds SELU/CELU to special set. |
| backends/arm/_passes/decompose_elu_pass.py | Adds ELU-family conversion pass and extends ELU decomposition to handle SELU/CELU + extra params. |
| backends/arm/_passes/convert_elu_params.py | Adjusts quantized ELU parameter handling to avoid int8 kernel crashes while preserving float params via metadata. |
| backends/arm/_passes/arm_pass_manager.py | Inserts the new ELU-family conversion pass into both backend and annotation pipelines. |
| backends/arm/_passes/init.py | Exposes the new ConvertEluFamilyToEluPass for pipeline construction. |
Comments suppressed due to low confidence (1)
backends/arm/_passes/decompose_elu_pass.py:33
get_elu_decomposition’s docstring still describes the old ELU rewrite (and doesn’t mentionscale/input_scaleorexpm1). Please update the docstring so it matches the implementation and the updated decomposition used inDecomposeEluPass.
def get_elu_decomposition(op) -> tuple:
"""Returns the decomposition of the given aten.elu operation into its
equivalent TOSA-supported operations.
This handles both edge dialect ops and core PyTorch ops. The decomposition strategy
is:
elu(x, y) → where(greater_or_eq(x, 0), (exp(x)-1), x)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+17
to
+23
| def _get_elu_parameter(args, kwargs, index, name): | ||
| if len(args) > index: | ||
| return args[index] | ||
|
|
||
| return kwargs.get(name, 1.0) | ||
|
|
||
|
|
| ) | ||
| replace_node.kwargs = updated_kwargs | ||
|
|
||
| # Save corret parameters |
Comment on lines
487
to
495
| ops_to_not_decompose_if_fp = { | ||
| torch.ops.aten.celu.default, | ||
| torch.ops.aten.eye.default, | ||
| torch.ops.aten.logit.default, | ||
| torch.ops.aten.linear.default, | ||
| torch.ops.aten.linspace.default, | ||
| torch.ops.aten.pad.default, | ||
| torch.ops.aten.selu.default, | ||
| } |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
backends/arm/_passes/decompose_elu_pass.py:33
- The get_elu_decomposition() docstring no longer matches the implementation: it claims to handle core PyTorch ops, and the decomposition formula shown is outdated (it omits scale/input_scale and uses a different branch ordering). Please update the docstring to reflect the current supported ops and decomposition used by DecomposeEluPass.
def get_elu_decomposition(op) -> tuple:
"""Returns the decomposition of the given aten.elu operation into its
equivalent TOSA-supported operations.
This handles both edge dialect ops and core PyTorch ops. The decomposition strategy
is:
elu(x, y) → where(greater_or_eq(x, 0), (exp(x)-1), x)
Comment on lines
9
to
12
| from executorch.backends.arm._passes import ArmPass | ||
| from executorch.backends.arm._passes.arm_pass_utils import create_node | ||
| from executorch.backends.arm._passes.insert_table_ops import InsertTableOpsPass | ||
| from executorch.backends.arm.constants import DQ_OPS |
Comment on lines
+17
to
+21
| def _get_elu_parameter(args, kwargs, index, name): | ||
| if len(args) > index: | ||
| return args[index] | ||
|
|
||
| return kwargs.get(name, 1.0) |
| ) | ||
| replace_node.kwargs = updated_kwargs | ||
|
|
||
| # Save corret parameters |
zingo
approved these changes
May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Arm backend: Improve ELU support
to ELU with particular scales.
rather than rounded values.