Skip to content

Add llm_methods metadata to multimethod PTE export#18773

Open
kimishpatel wants to merge 1 commit intopytorch:mainfrom
kimishpatel:export-D99689421
Open

Add llm_methods metadata to multimethod PTE export#18773
kimishpatel wants to merge 1 commit intopytorch:mainfrom
kimishpatel:export-D99689421

Conversation

@kimishpatel
Copy link
Copy Markdown
Contributor

Summary:
Add phase-aware method metadata to PTE files during multimethod export.
When MethodConfig entries have a phase tag ("prefill" or "decode"), the
export pipeline writes llm_methods_prefill and llm_methods_decode
constant methods into the PTE. This enables the runtime to discover which
methods to call for each inference phase without hardcoding names.

Changes:

  • MethodConfig: add optional phase field
  • _build_yoco_multimethod_config: tag methods with phase="prefill"/"decode"
  • _export_llm_backbone_multimethod: collect phase-tagged method names
  • _lower_and_save_multimethod: accept extra_metadata parameter
  • constants.h: add kLlmMethodsPrefill/kLlmMethodsDecode keys

Differential Revision: D99689421

Summary:
Add phase-aware method metadata to PTE files during multimethod export.
When MethodConfig entries have a `phase` tag ("prefill" or "decode"), the
export pipeline writes `llm_methods_prefill` and `llm_methods_decode`
constant methods into the PTE. This enables the runtime to discover which
methods to call for each inference phase without hardcoding names.

Changes:
- MethodConfig: add optional `phase` field
- _build_yoco_multimethod_config: tag methods with phase="prefill"/"decode"
- _export_llm_backbone_multimethod: collect phase-tagged method names
- _lower_and_save_multimethod: accept extra_metadata parameter
- constants.h: add kLlmMethodsPrefill/kLlmMethodsDecode keys

Differential Revision: D99689421
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot bot commented Apr 8, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18773

Note: Links to docs will display an error until the docs builds have been completed.

❌ 15 New Failures, 2 Unrelated Failures

As of commit ce726f6 with merge base e638059 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 8, 2026
@meta-codesync
Copy link
Copy Markdown
Contributor

meta-codesync bot commented Apr 8, 2026

@kimishpatel has exported this pull request. If you are a Meta employee, you can view the originating Diff in D99689421.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant