[ET Device Support] Add ExecutorchBackendConfig flags for skipping H2D/D2H copies

[Gasoonjia]

Stack from ghstack (oldest at bottom):

• [ET Device Support] Extract shared device test utilities to reduce redundancy #18762
• [ET Device Support] Add device tensor helper functions to TensorPtr API #18761
• -> [ET Device Support] Add ExecutorchBackendConfig flags for skipping H2D/D2H copies #18760
• [ET Device Support] PropagateDevicePass inserts H2D/D2H copy ops at delegate boundaries #18730
• [ET Device Support] Define et_copy runtime h2d and d2h copy ops #18729
• [ET Device Support] Define AOT device copy ops registry #18728
• [ET Device Support] CudaAllocator: device memory allocator for CUDA backend #18477
• [ET Device Support] Module: allocate device memory for planned buffers #18476
• [ET Device Support] MemoryManager: add per-buffer device metadata #18475
• [ET Device Support] MethodMeta: expose per-buffer device placement API #18474
• [ET Device Support] DeviceMemoryBuffer RAII class for device memory lifetime management #18473
• [ET Device Support] Emitter reads non_const_buffer_device from graph meta #18472
• [ET Device Support] Device-aware memory planning: separate buffers per device type #18375
• [ET Device Support] Add NonConstBufferDevice schema for per-buffer device mapping #18330
• [ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry #17535
• [ET Device Support] Annotate device attributes of CUDA backend IO tensors cuda device #18080
• [ET Device Support] Parse device info from serialized tensor in tensor_parser #18328
• [ET Device Support] Propagate device info from TensorSpec into serialized Tensor #18079
• [executorch] Propagate device metadata from partitioner result onto TensorSpecs #18078

Add skip_h2d_for_method_inputs and skip_d2h_for_method_outputs config
flags to ExecutorchBackendConfig. These control whether PropagateDevicePass
skips inserting H2D/D2H copy ops at method I/O boundaries:

• skip_h2d_for_method_inputs: user provides GPU tensor directly
• skip_d2h_for_method_outputs: output stays on device for cross-method pipelines

Differential Revision: D99636778

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18760

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 62 New Failures, 6 Unrelated Failures

As of commit 8e55dc6 with merge base b5ae0b9 ():

NEW FAILURES - The following jobs have failed:

• Build Presets / apple (pybind) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• Build Presets / linux (pybind, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• Build Presets / windows (pybind) / build (gh)
  Process completed with exit code 1.
• periodic / test-models-linux (buck2, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 9f6afb996095afb3438ef2f26483283a0edac6da317bcac86e43179461d9f504 /exec failed with exit code 1
• periodic / test-models-linux (cmake, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 8d1b7216b0580732b121d53b5c31ead267376b7b5fd41adea8cf180e4aa40b4b /exec failed with exit code 1
• periodic / test-models-linux (cmake, vit, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 9dcb441086973596cbaefbac275a64dab1d7a64327a7cd0b4ddab58ae2e74c60 /exec failed with exit code 1
• pull / android / build-android (gh)
  Process completed with exit code 2.
• pull / test-coreml-bc-macos (macos-m1-stable) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / test-coreml-bc-macos (macos-m2-stable) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 1c100614d45e379c2105cd0a5868102636be5b257e8f8f77e83ff21b135224b4 /exec failed with exit code 2
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-lora-linux / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-lora-multimethod-linux / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-mcu-cortex-m-backend / linux-job (gh)
  undefined reference to executorch::runtime::get_device_allocator(executorch::runtime::etensor::DeviceType)'`
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ba7c36ee987948555a49d1ce311c11cf23c553dbad463f52b9cc2b97a44260c8 /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f7da07b01757e3fccbf7ca09511644f46fe524c9883599ba9c067ca55f3b6ec8 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t cddc28bf64a67481105c4c4bab94ad0c7cb26e810ee52477716c1b61a4f0413c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f5a4992ed9237cd9cb7b6088b6967803a8be2938c544509f64def36a98943b3 /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f06aec28fbb4cc900c79e5795f462e00040af647184db4adaae9eca188e7eb10 /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 660bb0ce950226bfa23a84ac308f227001e3f7fc3eae6b1f8217fdddde3408a8 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 50925a866b127f10df414d6299aac385d0014691d451504a7b6c362db372412b /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e9285b21937a3dac968392040e4c45984583eeb956d1b985ff154cb6fa58031d /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b977d86f1c25b90e3458b7c9335126e22825f8bb60184b097a6c50ee351039d6 /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 157358029f6b73762cff545394fb0bdbb8c183f5f4704339b7badde5ad90bcf0 /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 2d3d0d63d33f01a927d7ce3064f4226b17fc42b2aee2b3493db57813c66613c8 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 30bd9e505865f2e9f33ae9653e8d55cdd6a5a99b85f07d5ec378a9d0494fc891 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t f85f730930b4ddf0526d79437d024176d50ef0ea0c01d947e5a992ea3871431a /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
  RuntimeError: Command docker exec -t 50600fb78247ecf708493678d61984d358ac0018ea00f343a4c094c90428efbe /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t c7bc63bdbb68a37dd9350a5c0f5c1e5fdfa469ef9e7a607fdfed3f8a9fe6a4b7 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t cc0c0053556e098205d76ad1e477d3b408d65ece7ec88b5447d094fb8b03e60e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
  RuntimeError: Command docker exec -t eb383b7f20cb60067d0eb9e333491203aeca2cb65c421d1a0001c791e9e11e0e /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  test_mimi
• pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
  RuntimeError: Command docker exec -t c18c3cef8f609775f0199ebed7de3197eeaef877549fcdfc11bc0491526ba702 /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 69b3fb47c54421a15cb8810325f311b1f8828b378fcf8b23edc0a8512a6105ec /exec failed with exit code 1
• pull / test-parakeet-xnnpack-linux / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
  AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
• pull / test-vulkan-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t ca95343b38cf6c1ac0303709ea6c7a11328a425a1f925845fed06e681b1105b4 /exec failed with exit code 1
• pull / test-vulkan-operators-linux / linux-job (gh)
  undefined reference to executorch::runtime::DeviceMemoryBuffer::create(unsigned long, executorch::runtime::etensor::DeviceType, signed char)'`
• pull / unittest / linux / linux-job (gh)
  backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
• pull / unittest / macos / macos-job (gh)
  backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
• pull / unittest / windows / windows-job (gh)
  Process completed with exit code 1.
• pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t e72c87cb7f1493decd4224423fd55e9e52d61614321359a0a337a9e29274f890 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-deps (test_pytest_ops_no_target) / linux-job (gh)
  RuntimeError: Command docker exec -t 13d1f6c39b9cee03ccf0a5269eb9c15e941665cca89ca93dea1c398e64edf658 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-deps (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 7306a11c7ba888af49efab8ffcb494483c7909ac65ec25d9cb9ec14cc17f645b /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
• pull / unittest-editable / macos / macos-job (gh)
  backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
• pull / unittest-editable / windows / windows-job (gh)
  Process completed with exit code 1.
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t e5a8c218bb09fcb867c553822d069dbc5700310fb69c18e57bdf6fa4b14b95cc /exec failed with exit code 2
• pull / unittest-wasm-bindings (--enable-etdump) / linux-job (gh)
  RuntimeError: Command docker exec -t f06ed275899ef67c6fa2ed449d0ff2a2c7449fe560d6e1589168e61f7d653422 /exec failed with exit code 2
• pull / unittest-wasm-bindings / linux-job (gh)
  RuntimeError: Command docker exec -t efe9e42550f3d02534f52e6599e6fdda48f59cb6a8a03d8734ae1117c46c7441 /exec failed with exit code 2

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-build-wasm-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Test ARM Backend / test-arm / test-backend-linux (arm_vgf_fp, operators) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Test QNN Backend / test-qnn / test-backend-linux (qnn, models) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Test QNN Backend / test-qnn / test-backend-linux (qnn, operators) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.