recipe(multilingual-e5-small): add CPU fp32/fp16 feature-extraction and sentence-similarity recipes

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Summary

Adds curated CPU float recipes for intfloat/multilingual-e5-small — a multilingual BertModel sentence embedder (E5) used for retrieval / semantic similarity (config model_type = bert, architectures = [BertModel], hidden 384, vocab 250037, type_vocab_size = 2 → has token_type_ids). Ships fp32 + fp16 for both feature-extraction and sentence-similarity under cpu/cpu/ (all quant: null; no CPU quantized variant per repo convention).

This is not a catalog clone: the default winml build (no recipe) FAILS at quantize for this model — the static-quant calibration reader feeds only input_ids + attention_mask, so quantize_static raises ValueError: Required inputs (['token_type_ids']) are missing. The quant: null float recipes skip that broken pass and build a working float model, and add an L2 numeric + retrieval-ranking parity proof that the trained embedding head is preserved end-to-end. Claimed tiers: Effort L0★ · Goal ceiling L2 · Outcome L0.

---

1. Recipe path(s)

• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp32_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp16_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp32_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp16_config.json

Per task, fp32 and fp16 are byte-identical (git blob OID 49f99b77… for feature-extraction, cda4592e… for sentence-similarity; quant: null float bucket — on CPU both realize as fp32, fp16 materializes on GPU/NPU). opset 17, batch 1, inputs input_ids[1,512] + attention_mask[1,512] + token_type_ids[1,512] (all int32), output last_hidden_state[1,512,384].

2. README row

None. Recipe-only, CPU-only — deliberately not added to the "passes fp16 eval on all 10 (EP, device) buckets" table, which would overclaim for a CPU-only recipe (consistent with #1084 / #1112 / #1117 / #1170).

3. Build output dir

temp/e5_fe/ (feature-extraction) and temp/e5_ss/ (sentence-similarity) — scratch, gitignored. model.onnx + model.onnx.data (448.25 MB float).

4. Build log

• Recipe FE: ✅ Build complete in 109.4s (Export 50.5 + Optimize 58.0; no quantize — quant: null).
• Recipe SS: ✅ Build complete in 98.3s (same graph, output last_hidden_state[1,512,384]).
• ONNX IR 8, opset 17, external data co-located.

5. Appended findings

model_knowledge/bert.json → new family file, bert-001 (baseline quantize-fails → quant:null fix; E5 token_type_ids gotcha; L2 parity numbers). Lane A (skill repo); not part of this model PR's diff.

6. Optimum-coverage probe

bert feature-extraction is VENDOR-ONLY (added_by_winml = []; vendor onnx tasks [feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, token-classification]). sentence-similarity is not a distinct Optimum onnx task — winml's task layer maps it onto the same AutoModel backbone export (the SS recipe builds the identical graph, confirmed). No winml-added exporter exists or is needed → L0★.

7. Claimed (Effort, Goal, Outcome)

L0★ / L2 / L0. Baseline gate: winml build -m intfloat/multilingual-e5-small (no -c) on main (origin/main = d9feefff, winml 0.2.0) — Export ✅ (61.2s, 448.3 MB) + Optimize ✅ (31.1s), then Quantize ❌ ValueError: Required inputs (['token_type_ids']) are missing from input feed (['input_ids', 'attention_mask']). The recipe fixes this exact failure by shipping quant: null float variants. Baseline HEAD == current origin/main (git rev-list --count HEAD..origin/main = 0).

8. Goal-ladder verdict table

Tier	Verdict	Evidence
L0 (build)	PASS	recipe-driven float build exit 0 (FE 109.4 s / SS 98.3 s, no quantize); ONNX IR 8 opset 17, inputs input_ids/attention_mask/token_type_ids [1,512] int32, output last_hidden_state[1,512,384], external data co-located
L1 (perf)	PASS	CPU/fp32 avg 256.26 ms, P50 241.30 ms, 3.90 samples/s, RAM Δ +186.9 MB
L2 (numeric vs PyTorch)	PASS	6 multilingual E5 texts: raw last_hidden_state max-abs 1.729e-06; mean-pooled+L2-normalized sentence-embedding cosine 1.000000; query→passage retrieval ranking identical (PyTorch == ONNX for both queries)

Ceiling L2 reached; no downgrade.

9. Methodology-evolution declaration

No methodology friction observed. The Step 1b baseline gate worked exactly as designed — it surfaced the quantize failure, which the quant: null CPU recipe fixes. This is the first bert-family model; bert.json / bert-001 is model knowledge, not a skill_meta change. No SKILL.md / agents/*.md edits needed.

10. Perf & eval data

EP / Device	Precision	Verdict	Mean	P50	Throughput	RAM Δ	Task metric
CPUExecutionProvider / cpu	fp32	PASS	256.26 ms	241.30 ms	3.90 samples/s	+186.9 MB	N/A (L3 not marched — L2 ceiling; English STS eval not meaningful for a multilingual E5 model without query/passage prefixes)

winml eval not run (Goal ceiling L2). Target EPs = [cpu]; coverage partial (GPU/NPU host-unreachable here).

11. Component / op-level data

winml analyze --ep all: 384 total operators, 15 unique types, all supported on the analyzed EP (384/0/0/0). Artifact: temp/e5_fe/analyze_all.json. (VitisAI all-unknown → analyze exit 1, expected, not a functional failure.)

12. Reproducible commands

# baseline (main): Export+Optimize pass, Quantize FAILS on missing token_type_ids
winml build -m intfloat/multilingual-e5-small -o temp\e5_baseline

# recipe-driven float builds (this PR)
winml build -m intfloat/multilingual-e5-small -c examples\recipes\intfloat_multilingual-e5-small\cpu\cpu\feature-extraction_fp32_config.json -o temp\e5_fe
winml build -m intfloat/multilingual-e5-small -c examples\recipes\intfloat_multilingual-e5-small\cpu\cpu\sentence-similarity_fp32_config.json -o temp\e5_ss

# L1 perf (CPU, pinned)
winml perf -m temp\e5_fe\model.onnx --device cpu --ep cpu --iterations 15 --warmup 3 --no-analyze

# op coverage
winml analyze -m temp\e5_fe\model.onnx --ep all -o temp\e5_fe\analyze_all.json

# L2 embedding + ranking parity vs PyTorch: temp\e5_l2.py
#   (E5 query:/passage: prefixes, pad to 512, int32 ids, token_type_ids=zeros since
#    XLMRobertaTokenizer emits none, mean-pool + L2 normalize, compare logits + argsort)
python temp\e5_l2.py

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

APPROVE

coverage: partial
reachable-verified: [cpu]
deferred: [gpu, npu, full-fp16-eval-matrix]

Checklist evidence:

• Fresh PR-head checkout: git rev-parse HEAD = 59530534c0aef5d65d2bb197301cefebce1daf39; merge-base/current origin/main = d9feefffccecfe47f08b7d2cfffb11fe57eb94df; git rev-list --count HEAD..origin/main = 0.
• Diff scope: git diff --name-only origin/main...HEAD is exactly 4 files: examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/{feature-extraction_fp16_config.json,feature-extraction_fp32_config.json,sentence-similarity_fp16_config.json,sentence-similarity_fp32_config.json}; no src/, tests, README, or skill files.
• Float CPU recipe set: both fp32/fp16 present for feature-extraction and sentence-similarity; no w8a16/w8a8 under cpu/cpu/ (git diff --name-only list above).
• Blob identity: git rev-parse HEAD:<path> gives FE fp32==fp16 49f99b779cc4f475b04c5e1e84059419a25cd299; SS fp32==fp16 cda4592eb8f5452f8ccf135e890ea881c846af1a.
• Recipe schema: Python JSON audit reports keys_ok True, quant_is_None True, opset 17, batch 1, loader {task, model_class: AutoModel, model_type: bert}, inputs input_ids/attention_mask/token_type_ids int32 [1,512], output last_hidden_state, clamp True for all 4.
• Baseline gate: temp\e5_baseline_build.log shows Export ✅ 61.2s / Optimize ✅ 31.1s then ValueError: Required inputs (['token_type_ids']) are missing from input feed (['input_ids', 'attention_mask']); temp\e5_baseline has export.onnx + optimized.onnx and no model.onnx.
• L0 artifacts: Python ONNX audit on temp\e5_fe\model.onnx and temp\e5_ss\model.onnx reports IR 8, opset 17, inputs input_ids, attention_mask, token_type_ids all elem_type 6 [1,512], output last_hidden_state [1,512,384]; .data files are co-located.
• L1 perf sanity: winml perf -m temp\e5_fe\model.onnx --device cpu --ep cpu --iterations 3 --warmup 1 --no-analyze reports CPUExecutionProvider fp32 avg 201.34 ms, throughput 4.97 samples/sec, RAM total +179.3 MB, consistent with PR’s 256.26 ms / 3.90 samples/sec / +186.9 MB.
• L2 parity: python temp\e5_l2.py reports raw last_hidden_state max-abs 1.729e-06, sentence embedding cosine min/mean 1.000000, query top-3 rankings [0,1,3] and [3,4,5] match PyTorch==ONNX, VERDICT: PASS; script lines 45-52 feed zero token_type_ids for XLMRobertaTokenizer single-segment default.
• Optimum/Effort tier: probe reports feature-extraction in vendor: True | added_by_winml: [] for bert, so L0★ recipe-only is correct; sentence-similarity maps to the same AutoModel backbone.
• PR body: gh api repos/microsoft/winml-cli/pulls/1182 body contains all 12 explainer sections, baseline FAIL→recipe FIX delta, L0/L1/L2 table, methodology declaration “No methodology friction observed”, CPU perf row, and README row omitted with CPU-only justification.
• Knowledge capture: Lane A model_knowledge\bert.json has bert-001, validated_on entries for FE/SS fp32 CPU, baseline error, recipe shape, L1/L2 numbers, analyze 384 ops / 15 unique, and deferred GPU/NPU/fp16 notes; it is not in this PR diff.
• Build artifacts read directly: temp\e5_fe/temp\e5_ss contain analyze_result.json, export_htp_metadata.json, and winml_build_config.json; audit shows 117,653,760 parameters, 228 modules, 88 traced, quant None, loader tasks correct, and optim clamp/fusions.
• Static gate: GitHub lint check succeeded; log shows uv run pre-commit run insert-license --all-files Passed, uv run ruff check src/ tests/ All checks passed, and uv run mypy -p winml.modelkit Success.
• N/A checks: Outcome-L1/L2 code/task-family add-ons, quant-preserving checks, composite gates, L3, NPU/GPU --ep-options, and skill_meta dead-link checks are not applicable because the diff is recipe-only L0★ with quant:null, no code/task/skill changes, and Goal ceiling L2.

Residual note: the test (models) check is red on an unrelated existing export unit (tests/unit/export/test_pytorch_export.py::TestExportPytorch::test_mismatched_input_order_exports_successfully); this PR changes only recipe JSONs and the required static gate is green.