Skip to content

Arm backend: Give SiLU its own output quantization spec - #21437

Open
vacu9708 wants to merge 1 commit into
pytorch:mainfrom
vacu9708:silu
Open

Arm backend: Give SiLU its own output quantization spec#21437
vacu9708 wants to merge 1 commit into
pytorch:mainfrom
vacu9708:silu

Conversation

@vacu9708

@vacu9708 vacu9708 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

SiLU's quantized output is forced to reuse its input's quantization scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.

Where this happens:

Quantizer Mechanism silu.default silu_.default
Composable (_TOSAQuantizerV2, default since #19758) TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS shared (bug) shared (bug)
Legacy (_TOSAQuantizerV1) quantization_annotator.py op sets own qspec shared (bug)

Why quantization sharing is wrong for SiLU:

image
  • SiLU non-linearly compresses and warps its input range:
    • Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for non-negative inputs, SiLU is non-linear across its entire domain.
    • For negative inputs, its output never goes below ~0.278, no matter how negative the input gets. So its output range is always narrower than its input's. Reusing the input scale misaligns the quantization bins with the actual output distribution, wasting resolution.
  • Reusing the input's scale doesn't save any computation at runtime: SiLU runs as a lookup table (class TableOps) whose entries are precomputed at compile time, not runtime.

Changes

File Change Why
quantizer/quantization_config.py Remove silu.default/silu_.default from SHARED_OUTPUT_ACT_QSPEC_PATTERNS The accuracy loss as described above.
quantizer/quantization_annotator.py Move silu_.default into _one_to_one, next to silu.default The legacy annotator classified the two variants differently. #17202 put silu.default in _one_to_one (own qspec) but silu_.default in _one_to_one_shared_input_or_input_act_qspec (shared qspec).
test/ops/test_silu.py Silu.forward clones its input Latent bug found while working on this change: SiLU(inplace=True) was mutating the test pipeline's shared input tensor which the pipeline reuses. It stayed invisible because the buggy (too large) output scale made the comparison helper(compare_rel_frobenius_and_cosine_similarity()) treat every value as noise and skip the check entirely, so the test kept passing without validating anything.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

Issue:
1. SiLU's quantized output was forced to reuse its input's quantization
scale, wasting resolution since SiLU's output range is always
narrower than its input's.
2. The composable quantizer and `silu_.default` was
separately misclassified in the legacy annotator since pytorch#17202.

Fix:
Give SiLU its own output qspec.
Also fixes an unrelated latent bug that this change exposed.

Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
@vacu9708
vacu9708 requested a review from digantdesai as a code owner July 28, 2026 18:41
@pytorch-bot

pytorch-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21437

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 12 Awaiting Approval

As of commit fe4a272 with merge base 14dab10 (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 28, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Jul 28, 2026
@pytorch-bot

pytorch-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:

  • ciflow/trunk

Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants