Add llm_methods metadata to multimethod PTE export#18773
Add llm_methods metadata to multimethod PTE export#18773kimishpatel wants to merge 1 commit intopytorch:mainfrom
Conversation
Summary:
Add phase-aware method metadata to PTE files during multimethod export.
When MethodConfig entries have a `phase` tag ("prefill" or "decode"), the
export pipeline writes `llm_methods_prefill` and `llm_methods_decode`
constant methods into the PTE. This enables the runtime to discover which
methods to call for each inference phase without hardcoding names.
Changes:
- MethodConfig: add optional `phase` field
- _build_yoco_multimethod_config: tag methods with phase="prefill"/"decode"
- _export_llm_backbone_multimethod: collect phase-tagged method names
- _lower_and_save_multimethod: accept extra_metadata parameter
- constants.h: add kLlmMethodsPrefill/kLlmMethodsDecode keys
Differential Revision: D99689421
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18773
Note: Links to docs will display an error until the docs builds have been completed. ❌ 15 New Failures, 2 Unrelated FailuresAs of commit ce726f6 with merge base e638059 ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@kimishpatel has exported this pull request. If you are a Meta employee, you can view the originating Diff in D99689421. |
This PR needs a
|
Summary:
Add phase-aware method metadata to PTE files during multimethod export.
When MethodConfig entries have a
phasetag ("prefill" or "decode"), theexport pipeline writes
llm_methods_prefillandllm_methods_decodeconstant methods into the PTE. This enables the runtime to discover which
methods to call for each inference phase without hardcoding names.
Changes:
phasefieldDifferential Revision: D99689421