NXP backend: Update docs for ET 1.4 release

backends/nxp/README.md

@@ -62,4 +62,4 @@ If you have problems or questions or have suggestions for ways to make
 implementation and testing better, please reach out to the NXP representative for the SoC you are interested in using,
 or your distribution partner contact.
 
-Or raise the issue here on ExecuTorch GitHub, label it with `module: nxp` and our ML team will address it on a priority-basis.
+Or raise the issue here on ExecuTorch [Github](https://github.com/pytorch/executorch/issues/new), label it with `module: nxp` and our ML team will address it on a priority-basis.

backends/nxp/neutron_partitioner.py

@@ -345,6 +345,8 @@ def __init__(
                                               `edge_program` only contains fake tensors without any data. In this case,
                                               this state dict is used instead (if provided). Notice: It may potentially
                                               contain outdated data,
+        :param preserve_ops: List of aten operators to not decompose during the lowering.
+        :param check_op_support: Optional callable to check if an operator is supported.
         """
         super().__init__()
         self.delegation_spec = DelegationSpec(NeutronBackend.__name__, compile_spec)

backends/nxp/nxp_backend.py

@@ -82,7 +82,7 @@ def neutron_compile_spec(
         :param config: Neutron accelerator configuration, e.g. "imxrt700"
         :param intermediates_dir: Directory to store intermediate artifact files.
         :param extra_flags: Extra flags for the Neutron compiler
-        :param operators_not_to_delegate: List of operators that should not be delegated
+        :param operators_not_to_delegate: List of operators that will not be delegated
         :param use_neutron_for_format_conversion: If True, the EdgeProgramToIRConverter will insert `Transpose` ops to
                                                 ensure that the IO matches the executorch partition, which will be
                                                 delegated to Neutron.

docs/source/backends/nxp/nxp-overview.md

@@ -11,7 +11,7 @@ For up-to-date status about running ExecuTorch on Neutron backend please visit t
 ## Features
 
 
-ExecuTorch v1.2 supports running machine learning models on selected NXP chips (for now only `i.MXRT700`).
+ExecuTorch v1.4 supports running machine learning models on selected NXP chips (for now only `i.MXRT700`).
 Among currently supported machine learning models are:
 - Convolution-based neutral networks
 - Full support for MobileNetV2 and CifarNet

docs/source/backends/nxp/nxp-partitioner.rst

@@ -9,7 +9,12 @@ It has the following arguments:
 * `compile_spec` - list of key-value pairs defining compilation,
 * `neutron_target_spec` - NeutronTargetSpec instance, initialized by SoC id, e.g. "imxrt700",
 * `custom_delegation_options` - custom options for specifying node delegation,
-* `preserve_ops` - list of aten operators to not be decomposed by ExecuTorch.
+* `post_quantization_state_dict` - state-dict of the model right after quantization. During partitioning, the
+                                                `edge_program` only contains fake tensors without any data. In this case,
+                                                this state dict is used instead (if provided). Notice: It may potentially
+                                                contain outdated data,
+* `preserve_ops` - list of aten operators to not decompose during the lowering.
+* `check_op_support` - optional callable to check if an operator is supported.
 
 --------------------
 Compile Spec Options
@@ -18,11 +23,13 @@ To generate the Compile Spec for Neutron backend, you can use the `generate_neut
 Following fields can be set:
 
 * `config` - NXP platform defining the Neutron NPU configuration, e.g. "imxrt700".
+* `intermediates_dir` - Directory to store intermediate artifact files.
 * `extra_flags` - Extra flags for the Neutron compiler.
 * `operators_not_to_delegate` - List of operators that will not be delegated.
 * `use_neutron_for_format_conversion` - If True, let the eIQ Neutron NPU to handle conversion between channel-first (NCHW) and channel-last (NHWC) data formats. That is the Neutron backend will insert `Transpose` ops to ensure that the IO matches the executorch partition, which will be delegated to Neutron.
-* `fetch_constants_to_sram`: If True, the Neutron Converter will insert microinstructions to prefetch weights from FLASH to SRAM. This should be used when the whole model does not fit into SRAM on Neutron-C devices, like i.MX RT700
-* `dump_kernel_selection_code`: Whether Neutron converter dumps kernel selection code, which is used by the selective kernel registration, see :doc:`Neutron Firmware Kernel Selection support <nxp-kernel-selection.md>`.
+* `fetch_constants_to_sram` - If True, the Neutron Converter will insert microinstructions to prefetch weights from FLASH to SRAM. This should be used when the whole model does not fit into SRAM on Neutron-C devices, like i.MX RT700.
+* `dump_kernel_selection_code` - Whether Neutron converter dumps kernel selection code, which is used by the selective kernel registration, see :doc:`Neutron Firmware Kernel Selection support <nxp-kernel-selection.md>`.
+* `use_profiling` - If true Neutron Converter will enable profiling for neutron delegated model.
 
 -------------------------
 Custom Delegation Options
@@ -37,7 +44,7 @@ Operator Support
 Operators are the building blocks of the ML model. See `IRs <https://docs.pytorch.org/docs/stable/torch.compiler_ir.html>`_ for more information on the PyTorch operator set.
 
 This section lists the Edge operators supported by the Neutron backend.
-For detailed constraints of the operators see the ``is_supported`` / ``_is_supported_in_IR`` / ``_is_supported_on_target`` checks in the `Node converters <https://github.com/pytorch/executorch/blob/main/backends/nxp/backend/ir/converter/node_converter.py#L118>`_
+For detailed constraints of the operators see the ``is_supported`` / ``_is_supported_in_IR`` / ``_is_supported_on_target`` checks in the `Node converters <https://github.com/pytorch/executorch/blob/main/backends/nxp/backend/ir/converter/node_converter.py#L105>`_
 
 
 .. csv-table:: Operator Support

docs/source/backends/nxp/nxp-quantization.md

@@ -11,29 +11,55 @@ The Neutron delegate supports the following quantization schemes:
     - Following operators are supported at this moment: 
       - `aten.abs.default`
       - `aten.adaptive_avg_pool2d.default`
-      - `aten.addmm.default`
       - `aten.add.Tensor`
+      - `aten.addmm.default`
+      - `aten.amax.default`
+      - `aten.amin.default`
+      - `aten.avg_pool1d.default`
       - `aten.avg_pool2d.default`
+      - `aten.batch_norm.default`
+      - `aten.bmm.default`
       - `aten.cat.default`
-      - `aten.conv1d.default`
+      - `aten.clamp.default`
       - `aten.conv2d.default`
+      - `aten.conv2d.padding`
+      - `aten.conv_transpose2d.input`
       - `aten.dropout.default`
+      - `aten.exp.default`
       - `aten.flatten.using_ints`
       - `aten.hardtanh.default`
       - `aten.hardtanh_.default`
+      - `aten.leaky_relu.default` and `aten.leaky_relu_.default`
       - `aten.linear.default`
+      - `aten.log.default`
+      - `aten.maximum.default`
+      - `aten.max_pool1d.default`
       - `aten.max_pool2d.default`
       - `aten.mean.dim`
+      - `aten.minimum.default`
+      - `aten.mm.default`
       - `aten.mul.Tensor`
+      - `aten.neg.default`
       - `aten.pad.default`
       - `aten.permute.default`
+      - `aten.prelu.default`
       - `aten.relu.default` and `aten.relu_.default`
       - `aten.reshape.default`
-      - `aten.view.default`
-      - `aten.softmax.int`
-      - `aten.tanh.default`,  `aten.tanh_.default`
       - `aten.sigmoid.default`
-      - `aten.slice_copy.Tensor`
+      - `aten.slice.Tensor`
+      - `aten.softmax.int`
+      - `aten.squeeze.default`
+      - `aten.squeeze.dim`
+      - `aten.squeeze.dims`
+      - `aten.sub.Tensor`
+      - `aten.sum.default`
+      - `aten.sum.dim_IntList`
+      - `aten.tanh.default` and `aten.tanh_.default`
+      - `aten.transpose.int`
+      - `aten.unsqueeze.default`
+      - `aten.upsample_bilinear2d.vec`
+      - `aten.upsample_nearest2d.vec`
+      - `aten.view.default`
 
 ### Static 8-bit Quantization Using the PT2E Flow
 

docs/source/backends/nxp/op-support.csv

@@ -2,47 +2,48 @@ Operator,Compute DType,Quantization,Constraints
 aten.abs.default,int8,static int8,
 aten._adaptive_avg_pool2d.default,int8,static int8,"ceil_mode=False, count_include_pad=False, divisor_override=False"
 aten.addmm.default,int8,static int8,2D tensor only
-aten.add.Tensor,int8,static int8,"alpha = 1, input tensors of equal shape"
+aten.add.Tensor,int8,static int8,"alpha = 1"
 aten.amax.default,int8,static int8,
 aten.amin.default,int8,static int8,
 aten.avg_pool1d.default,int8,static int8,"ceil_mode=False, count_include_pad=False, divisor_override=False"
 aten.avg_pool2d.default,int8,static int8,"ceil_mode=False, count_include_pad=False, divisor_override=False"
-aten.bmm.default,int8,static int8,"width and channels dim of both args %8 = 0, 3D tensors only"
-aten.cat.default,int8,static int8,"input_channels % 8 = 0, output_channels %8 = 0"
+aten.bmm.default,int8,static int8,"quantization zero points of inputs are equal"
+aten.cat.default,int8,static int8,"all inputs share quantization parameters"
 aten.clamp.default,int8,static int8,"Bounds = (-1, 1) or (0, 1) or (0, 6) or (0, None)"
-aten.clone.default,int8,static int8,
+aten.clone.default,int8,static int8,"node has memory format specified"
 aten.constant_pad_nd.default,int8,static int8,"H or W padding only"
 aten.convolution.default,int8,static int8,"1D or 2D convolution, constant weights, groups=1 or groups=channels_count (depthwise)"
 aten.dim_order_ops._clone_dim_order.default,,, "See aten.clone.default"
 aten.div.Tensor,int8,static int8,"divisor - static tensor or scalar value, one dimension must satisfy %8 = 0 or scalar division (all dims = 1)"
 aten.exp.default,int8,static int8,
-aten.hardtanh.default,int8,static int8,"supported ranges: <0,6>, <-1, 1>, <0,1>, <0,inf>"
+aten.hardtanh.default,int8,static int8,"Bounds = (-1, 1) or (0, 1) or (0, 6) or (0, None)"
 aten.leaky_relu.default,int8,static int8,
 aten.log.default,int8,static int8,
-aten.max_pool1d.default,int8,static int8,"dilation=1, ceil_mode=False, channels%8=0, batch_size=1, stride_h=1 or 2"
-aten.max_pool2d.default,int8,static int8,"dilation=1, ceil_mode=False, channels%8=0, batch_size=1, stride_h=1 or 2"
-aten.max_pool2d_with_indices.default,int8,static int8,"dilation=1, ceil_mode=False, channels%8=0, batch_size=1, stride_h=1 or 2"
+aten.max_pool1d.default,int8,static int8,"dilation=1, ceil_mode=False, stride < 4096"
+aten.max_pool2d.default,int8,static int8,"dilation=1, ceil_mode=False, stride < 4096"
+aten.max_pool2d_with_indices.default,int8,static int8,"dilation=1, ceil_mode=False, stride < 4096"
 aten.maximum.default,int8,static int8,
-aten.mean.dim,int8,static int8,"4D tensor only, dims = [-1,-2] or [-2,-1]"
+aten.mean.dim,int8,static int8,"4D tensor only"
 aten.minimum.default,int8,static int8,
 aten.mm.default,int8,static int8,"2D tensor only"
-aten.mul.Tensor,int8,static int8,"tensor-size % 8 = 0"
+aten.mul.Tensor,int8,static int8,
 aten.neg.default,int8,static int8,
 aten.permute_copy.default,int8, static int8, "Only specific transpositions supported, see backends/nxp/backend/ir/converter/node_converters/ops_converters/permute_copy_converter.py"
-aten.prelu.default,int8, static int8, "rank = 4, channels % 8 = 0, flat input size / channels <= 4096"
+aten.prelu.default,int8, static int8,
 aten.relu.default,int8,static int8,
 aten.sigmoid.default,int8,static int8,
-aten.slice_copy.Tensor,int8, static int8
+aten.slice.Tensor,int8, static int8
 aten._softmax.default,int8, static int8, "rank > 1, channels % 8 = 0, channels < 2048, flat input size / channels <= 4096, flat input size <= 524288"
 aten.split.default,N/A, N/A, "transforming split -> getitem to slice, see aten.slice_copy.Tensor"
 aten.split.Tensor,N/A, N/A, "transforming split -> getitem to slice, see aten.slice_copy.Tensor"
 aten.split_with_sizes.default,N/A, N/A, "transforming split -> getitem to slice, see aten.slice_copy.Tensor"
 aten.squeeze.default,int8,static int8,
 aten.squeeze.dim,int8,static int8,
 aten.squeeze.dims,int8,static int8,
+aten.sub.Tensor,int8,static int8,"alpha = 1"
 aten.sum.dim_IntList,int8,static int8,
 aten.tanh.default,int8,static int8,
 aten.unsqueeze.default,int8,static int8,
-aten.upsample_bilinear2d.vec,int8,static int8,"channels % 8 = 0, H_scale = W_scale = 2 or 4"
-aten.upsample_nearest2d.vec,int8,static int8,"channels % 8 = 0, H_scale = W_scale = 2 or 4"
+aten.upsample_bilinear2d.vec,int8,static int8,"H_scale and W_scale = 1, 2, 4 or 8"
+aten.upsample_nearest2d.vec,int8,static int8,"H_scale and W_scale = 1, 2, 4 or 8"
 aten.view_copy.default,int8,static int8,

docs/source/backends/nxp/tutorials/nxp-basic-tutorial.md

@@ -9,7 +9,7 @@ This tutorial demonstrates the use of the eIQ Neutron backend with the Neutron b
 ### Software
 First you need to have Python 3.10 - 3.12 installed. 
 
-You need to install the ExecuTorch. Please follow the tutorial to install the ExecuTorch [Setting Up ExecuTorch](../../../getting-started-setup.rst)
+You need to install the ExecuTorch. Please follow the tutorial to install the ExecuTorch [Getting Started with ExecuTorch](https://docs.pytorch.org/executorch/main/getting-started.html)
 
 
 In addition to this, you will need to install the eIQ Neutron Simulator, called NSYS,
@@ -31,7 +31,7 @@ format and delegate the model computation to eIQ Neutron NPU using the eIQ Neutr
 
 ### Step 1: Environment Setup
 
-This tutorial is intended to be run from a Linux and uses Conda or Virtual Env for Python environment management. For full setup details and system requirements, see [Getting Started with ExecuTorch](/getting-started).
+This tutorial is intended to be run from a Linux and uses Conda or Virtual Env for Python environment management. For full setup details and system requirements, see [Getting Started with ExecuTorch](https://docs.pytorch.org/executorch/main/getting-started.html).
 
 Create a Conda environment and install the ExecuTorch Python package.
 ```bash

examples/nxp/README.md

@@ -3,15 +3,17 @@ This directory contains examples demonstrating the use of ExecuTorch AoT flow to
 format and delegate the model computation to eIQ Neutron NPU using the eIQ Neutron Backend.
 
 ## Layout
+* `executor_runner` - contains utility to run compiled ExecuTorch programs on Neutron Backend.
 * `experimental/` - contains CifarNet model example.
 * `models` - demo models instantiation used in examples.
 * `aot_neutron_compile.py` - script with end-to-end ExecuTorch AoT Neutron Backend workflow.
 * `README.md` - this file.
+* `run.sh` - utility script to build nxp_executor_runner, convert example model and run it on NSYS.
 * `run_aot_example.sh` - utility script to launch _aot_neutron_compile.py_. Primarily for CI purpose. 
 * `setup.sh` - setup script to install Neutron Backend dependencies.
 
 ## Setup
-Please finish tutorial [Setting up ExecuTorch](https://pytorch.org/executorch/main/getting-started-setup).
+Please finish tutorial [Getting Started with ExecuTorch](https://docs.pytorch.org/executorch/main/getting-started.html).
 
 Run the setup.sh script to install the neutron-converter:
 ```commandline
@@ -43,4 +45,4 @@ The steps are expected to be executed from the `executorch` root folder.
 project, presented [here](https://mcuxpresso.nxp.com/mcuxsdk/latest/html/middleware/eiq/executorch/docs/nxp/topics/example_applications.html#how-to-build-and-run-executorch-cifarnet-example).
 This project will guide you through the process of deploying your PTE model to the device.
 To get the MCUXpresso SDK follow this [guide](https://mcuxpresso.nxp.com/mcuxsdk/latest/html/middleware/eiq/executorch/docs/nxp/topics/getting_mcuxpresso.html),
-use the MCUXpresso SDK v25.12.00. 
+use the MCUXpresso SDK v26.06.00.