fix(recipe): add fp16 quant config to cross-encoder/nli-deberta-v3-base recipe

[yueyong (codykk)]

Summary

This L0 recipe contribution makes the CPU fp16 recipe for cross-encoder/nli-deberta-v3-base materially request fp16 conversion for its three-way natural-language-inference classifier. The exact candidate reached Goal L3 PASS with full planned CPU fp32/fp16 coverage: both artifacts build and run, both have measured performance and PyTorch-logit parity, and exactly one bounded fp32 SNLI functional smoke completed. The shipped Effort / Goal / Outcome is L0 / L3 / L0; no shared code or evaluator behavior changed.

Model metadata

What the model does

This English cross-encoder performs three-way natural-language inference: it jointly tokenizes a premise and hypothesis and emits logits for contradiction, entailment, and neutral.

• Evidence/confidence: the model card and config.json at checkpoint revision 6c749ce3425cd33b46d187e45b92bbf96ee12ec7 identify zero-shot classification, MultiNLI/SNLI training data, and the authoritative labels contradiction, entailment, and neutral (verified).

Primary user stories

• A user supplies a premise and hypothesis to obtain whether the hypothesis contradicts, follows from, or is neutral toward the premise. Evidence: the pinned model card and DebertaV2ForSequenceClassification config with three NLI labels (verified).
• A user supplies a text and candidate statement to obtain an entailment score for bounded zero-shot classification. Evidence: the pinned model-card zero-shot-classification pipeline tag (inferred).

Supported tasks

Task	Support surfaces	Evidence	Confidence
text-classification	Transformers, Optimum ONNX, WinML	Pinned DebertaV2ForSequenceClassification config; Optimum's deberta-v2 registry; WinML inspect on main	verified
zero-shot-classification	checkpoint	Pinned model-card pipeline tag	verified

Model architecture

A DeBERTa-v2 encoder cross-encoder with 12 disentangled-relative-attention layers, first-token context pooling, and a three-logit NLI head.

DebertaV2ForSequenceClassification
+-- DebertaV2Model
|   +-- Embeddings (vocab 128100 -> hidden 768; position embeddings; no token-type embedding table)
|   +-- Encoder stack x 12
|       +-- Disentangled self-attention (12 heads; content/relative-position bias)
|       +-- Feed-forward (768 -> 3072 -> 768)
|       +-- Residual connections + LayerNorm
+-- ContextPooler (first token, 768 -> 768)
+-- Dropout + classification head (768 -> 3 logits)

• Source/confidence: pinned checkpoint config.json and the Transformers DebertaV2ForSequenceClassification / DebertaV2Model source (verified).

Validation and support evidence

Baseline

• Revision-2 refresh provenance: the baseline moved from main c45e9513964bc0bc55ad9cdd612537e96349c8a3 to e7b132d5f30c96c11ca818ef30d9b1ec6c867e15 under a PARTIAL-RERUN. The interval included the optimization/analyzer refactor merged by microsoft/winml-cli#1298, so config, build, structural validation, perf, Eval, Analyze, and quality gates were refreshed on e7b132d5; only the frozen model breakdown, model identity, and Eval-plan semantics were reused from c45e9513. Candidate build, perf, parity, Eval, Analyze, and quality evidence was then executed on exact candidate ac594976b0906a0fe5b17e98744a05932bd66c1c with parent/base e7b132d5.
• Revision-3 reuse confirmation: authoritative current main and the PR base remained e7b132d5f30c96c11ca818ef30d9b1ec6c867e15. The interval e7b132d5..e7b132d5 contained zero changed files, all dependency surfaces were unchanged, and the live candidate patch/tree/blob remained equivalent. Revision 3 therefore recorded REUSE with zero rerun stages and zero downstream invalidations. All measurements below remain the revision-2 measurements executed on e7b132d5 or exact candidate ac594976; revision 3 did not execute or relocate them.
• Version and recipe-free floor: source pyproject.toml reported WinML 0.3.0 while installed metadata reported 0.2.0. Recipe-free/no-precision CPU build passed ONNX checker at opset 17 with 620 nodes, INT32 input_ids and attention_mask [1,512], FLOAT logits [1,3], and 824,199,168 external-data bytes. The starting auto-config had quant: null.
• Perf floor: fp32 CPU mean 368.611 ms, p50 378.033 ms, throughput 2.71 samples/s, RSS total delta 275.54 MB.
• Eval floor: bounded fp32 CPU SNLI functional smoke passed with accuracy 1.0 on exactly two deterministic validation rows and latency 0.3483504999894649 s/sample. Dataset IDs (entailment=0, neutral=1, contradiction=2) were aligned by authoritative names to checkpoint IDs (contradiction=0, entailment=1, neutral=2). This was operability evidence, not benchmark accuracy.
• Optimum probe: VENDOR-ONLY. Optimum already covered feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, and token-classification for deberta-v2; WinML added no exporter override.

Goal

• Effort: L0, recipe-only.
• Committed Goal ceiling: L3, requiring L0 build/structure, L1 perf, L2 PyTorch parity, and L3 bounded functional smoke in one ladder march.
• Outcome target: L0, checked-in exact-evidence recipes with no shared code change.
• Success definition: both planned cpu/cpu/fp32 and cpu/cpu/fp16 tuples build independently from their checked-in recipes; both retain expected structure and measured performance; both match PyTorch logits and classes on the bounded pair set; exactly one final-candidate fp32 CPU functional smoke proves evaluator operability. No ceiling re-issue or downgrade occurred.

Outcome

• Candidate: ac594976b0906a0fe5b17e98744a05932bd66c1c; shipped tier: L0; highest Goal: L3 PASS.
• Target coverage: full. Required/passed tuples: cpu/cpu/fp32, cpu/cpu/fp16. Deferred target tuples: []. Analyze-only unavailable accelerator runtime rows described below are not target-coverage deferrals.
• Recipe scope: the existing fp32 recipe remains authoritative and byte-for-byte unchanged; this PR changes only examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp16_config.json. No source, tests, evaluator behavior, or production recipe README changed.
• Knowledge handoff: DeBERTa findings for exact fp16 realization and parity, authoritative SNLI name remapping, and complete graph mapping with the QNN GatherElements static-rule limitation are isolated in separate Lane A draft PR gim-home/ModelKitArtifacts#211, exact head df47d1bf7a08dab528434cf2e0b7d98fc052862a, with one-file scope.

Quality gates

• Config/schema and licenses passed: checkpoint Apache-2.0; SNLI CC-BY-SA-4.0. License insertion, Ruff, and mypy passed; mypy reported no issues in 435 source files.
• The focused changed-optimization suite passed: 113 passed, 6 skipped.
• Five local partitions passed, 8,055 tests total: analyze 1,526 passed / 45 skipped; models/loader/datasets/export 1,523 passed / 6 skipped / 2 xfailed; optim 582 passed / 16 skipped / 1 xfailed; commands/config/build/compiler/session/eval 3,555 passed / 9 skipped / 1 warning; remaining unit/regression/CLI 869 passed / 2 skipped / 1 deselected / 1 warning. Across those partitions: 78 skipped, 3 xfailed, 1 deselected, and 2 warnings.
• Live exact-head checks passed 9/9: Analyze (Python), lint, test (analyze), test (models), test (optim), test (commands), test (remaining), CodeQL, and license/cla. The live diff remains exactly one fp16 recipe file.

Limitations

• Runtime build, performance, parity, and Eval validation was CPU-only. No GPU or NPU accelerator runtime, performance, or parity claim is made.
• Analyze returned ANALYZE-PARTIAL-SUCCESS with exit code 1 for both artifacts. All seven requested EP rows and optimization output were persisted; the nonzero exit reflects unavailable local provider runtime registration. CUDAExecutionProvider, MIGraphXExecutionProvider, QNNExecutionProvider, TensorrtExecutionProvider, and DmlExecutionProvider were unavailable locally. NvTensorRTRTXExecutionProvider and OpenVINOExecutionProvider had runtime_support=true rows, but these Analyze classifications are still static compatibility evidence, not execution of this model on those accelerators.

Per-EP/device/precision results and Functional smoke Eval

Tier	EP / device	Precision	Verdict	Artifact structure	Mean	p50	Throughput	RSS total delta
L0 / L1	CPUExecutionProvider / cpu	fp32	PASS / PASS	checker PASS; IR 8; opset 17; 568 nodes; 225 FLOAT + 59 INT64 initializers; 824,199,168 external bytes; INT32 [1,512] inputs; FLOAT [1,3] logits	332.598 ms	335.852 ms	3.01 samples/s	275.65 MB
L0 / L1	CPUExecutionProvider / cpu	fp16	PASS / PASS	checker PASS; IR 8; opset 17; 569 nodes; 225 FLOAT16 + 59 INT64 initializers; 437,265,408 external bytes; INT32 [1,512] inputs; FLOAT [1,3] logits	389.293 ms	393.143 ms	2.57 samples/s	285.38 MB

The fp32 and fp16 builds completed in 110.7863908 s and 124.9451469 s respectively. The fp16 external-data ratio was 0.5305336682892647; fp16 was materially realized rather than inferred from the filename or recipe declaration.

L2 logits parity: PASS on the same two premise/hypothesis pairs and all six logits per precision. fp32 had aggregate maximum absolute error 2.86102294921875e-06, cosine 1.0, and all predicted class names matched PyTorch. fp16 had aggregate maximum absolute error 0.000835418701171875, cosine 1.0, and all predicted class names matched PyTorch.

Functional smoke Eval (L3): exactly one r2 invocation on exact candidate ac594976b0906a0fe5b17e98744a05932bd66c1c, CPU fp32. Dataset stanfordnlp/snli revision cdb5c3d5eed6ead6e5a341c8e56e669bb666725b, config plain_text, validation split, CC-BY-SA-4.0; deterministic unshuffled first two source-order rows. Requested 2, selected 2, processed 2, filtered 0, failed 0. Fan-out was capped at one premise/hypothesis pair per row, static batch size 1, and sequence length 512; candidate-label/prompt, beam, frame, and crop caps were not applicable. The schema was premise: string, hypothesis: string, and label: ClassLabel(entailment, neutral, contradiction). Dataset label IDs 0,1,2 map by authoritative names to checkpoint IDs 1,2,0; argmax over the three logits is interpreted with checkpoint labels 0=contradiction, 1=entailment, 2=neutral. Accuracy was 1.0. This proves end-to-end operability only and is not representative accuracy or benchmark quality; no fp16 Eval was performed.

Delta

• The exact live diff is one file: examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp16_config.json, with 23 insertions and 1 deletion. JSON pointer /quant changes from null to this serialized object:

{
  "mode": "fp16",
  "samples": 10,
  "calibration_method": "minmax",
  "weight_type": "uint8",
  "activation_type": "uint8",
  "per_channel": false,
  "symmetric": false,
  "weight_symmetric": null,
  "activation_symmetric": null,
  "save_calibration": false,
  "distribution": "uniform",
  "seed": null,
  "calibration_load_path": null,
  "calibration_save_path": null,
  "op_types_to_quantize": null,
  "nodes_to_exclude": null,
  "task": "text-classification",
  "model_id": "cross-encoder/nli-deberta-v3-base",
  "model_type": "deberta-v2",
  "fp16_keep_io_types": true,
  "fp16_op_block_list": null
}

• The existing fp32 recipe is unchanged and retains quant: null. Relative to current-main fp16 auto-config, quant, compile, and loader match. The checked-in recipe preserves checkpoint-specific optim.clamp_constant_values=true and the existing eager-attention export/input policy.
• Relative to the pre-rebase candidate, the focused recipe diff is empty: the candidate patch and recipe content were preserved across the r2 current-main refresh. The change is intentionally per-model and not metadata-derivable, so recipe-free acceptance was not required. Reducibility remains consistent with the L0 charter.
• There are no code changes, no shared-logic checkpoint hardcoding, and no class-wide behavior change. examples/recipes/README.md remains untouched.

Analyze summary - component level and op level

Both artifacts are ANALYZE-PARTIAL-SUCCESS (exit 1): all seven expected EP rows, optimization output, and complete component mappings were persisted before unavailable local provider runtime registration caused the nonzero exit. Static rule analysis is compatibility guidance, not accelerator runtime execution.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP finding
fp32	encoder setup; embeddings (5 nodes); all 12 attention + FFN layers; pooler; inference-removed dropout (0 nodes); classifier	29 component records; 568 / 568 nodes mapped; 0 unmapped; 100% coverage	QNN partial: GatherElements; none unsupported
fp16	encoder setup; embeddings (5 nodes); all 12 attention + FFN layers; pooler; inference-removed dropout (0 nodes); classifier	29 component records; 569 / 569 nodes mapped; 0 unmapped; 100% coverage	QNN partial: GatherElements; none unsupported

Mappings were based on fresh optimized-graph source scopes plus contiguous fused-node ownership. No model component was omitted.

Op-level summary

Artifact	Graph	Dominant operators	Grouped EP outcomes
fp32	568 operators / 17 unique types	Reshape 192; Gemm 74; Transpose 72; MatMul 48; Add 48; GatherElements 24; Cast 2	NvTensorRTRTX + OpenVINO: 17 statically supported types; QNN: 16 supported, GatherElements partial, none unsupported; CUDA + MIGraphX + TensorRT + DML: runtime unavailable locally and no populated classifications
fp16	569 operators / 17 unique types	Reshape 192; Gemm 74; Transpose 72; MatMul 48; Add 48; GatherElements 24; Cast 3	Same grouped outcome as fp32

QNN reported runtime_support=false; therefore its 24 GatherElements nodes are a partial static-rule finding only, not QNN NPU execution, performance, or parity evidence. The unavailable Analyze accelerator rows do not alter the full CPU target coverage or create target tuple deferrals.

Reproduce commands

The recipes remain authoritative. The fp16 selector is the only required precision-selection argument; no recipe-owned task, model-class, shape, quantization, optimization, compilation, dataset, or sample setting is overridden.

$OUT='temp/cross-encoder-nli-deberta-v3-base'

# Build each checked-in recipe.
python -m winml.modelkit.cli build -c examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp32_config.json -m cross-encoder/nli-deberta-v3-base -o $OUT/fp32 --ep cpu --device cpu
python -m winml.modelkit.cli build -c examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp16_config.json -m cross-encoder/nli-deberta-v3-base -o $OUT/fp16 --ep cpu --device cpu --precision fp16

# Reproduce the reported performance protocol.
python -m winml.modelkit.cli perf -m $OUT/fp32/model.onnx --task text-classification --ep cpu --device cpu --warmup 3 --iterations 10 --memory -o $OUT/perf-fp32.json --format json --overwrite
python -m winml.modelkit.cli perf -m $OUT/fp16/model.onnx --task text-classification --ep cpu --device cpu --warmup 3 --iterations 10 --memory -o $OUT/perf-fp16.json --format json --overwrite

# Exactly one bounded final-artifact functional smoke Eval.
'{"contradiction":0,"entailment":1,"neutral":2}' | Set-Content -Encoding utf8 $OUT/label-mapping.json
python -m winml.modelkit.cli eval -m $OUT/fp32/model.onnx --model-id cross-encoder/nli-deberta-v3-base --task text-classification --ep cpu --device cpu --dataset stanfordnlp/snli --dataset-name plain_text --dataset-revision cdb5c3d5eed6ead6e5a341c8e56e669bb666725b --split validation --samples 2 --no-shuffle --streaming --column input_column=premise --column second_input_column=hypothesis --column label_column=label --label-mapping $OUT/label-mapping.json --skip-build -o $OUT/eval-fp32-snli.json --format json --overwrite

# Static component/op compatibility plus optimization analysis. Exit 1 can accompany complete output when local EP runtimes cannot register.
python -m winml.modelkit.cli analyze --model $OUT/fp32/model.onnx --ep all --check-optim --output $OUT/analyze-fp32.json
python -m winml.modelkit.cli analyze --model $OUT/fp16/model.onnx --ep all --check-optim --output $OUT/analyze-fp16.json

[ssss141414]

ACCEPTABLE

Reviewed exact candidate ac594976b0906a0fe5b17e98744a05932bd66c1c as independent reviewer ssss141414 against current main/base e7b132d5f30c96c11ca818ef30d9b1ec6c867e15.

• Scope and engineering judgment: correct L0 one-file recipe change. Optimum already owns DeBERTa-v2 text-classification export; the existing checkpoint-specific input/optimization policy is preserved, while the fp16 quantization declaration is intentionally per-model rather than a class-wide inference.
• Independent acceptance: a fresh exact-head build using the checked-in recipe, --precision fp16, and no recipe-owned CLI overrides produced 225 FLOAT16 initializers, 437,265,408 external-data bytes, named INT32 input_ids and attention_mask inputs [1,512], and FLOAT logits [1,3]. Real named-input inference predicted entailment under the pinned contradiction/entailment/neutral label semantics.
• Evidence audit: both CPU tuple perf series, fp32/fp16 PyTorch parity, the unique two-row fp32 SNLI smoke and authoritative label remap, both seven-row Analyze outputs, full 29-component graph mappings, the bounded QNN GatherElements static limitation, and all frozen manifests revalidated.
• Coverage: full for required cpu/cpu/fp32 and cpu/cpu/fp16; deferred target tuples: []. Accelerator rows are Analyze-only static limitations, not runtime coverage claims.
• Final gates: license, Ruff, mypy, and all five non-hardware pytest partitions passed locally; 9/9 exact-head GitHub checks passed. Pagination found 0 issue comments, 0 review comments, 0 prior reviews, 0 review threads, and 0 unresolved threads.

Residual limitation: runtime, performance, parity, and functional-smoke evidence is CPU-only. QNN GatherElements remains static partial-support evidence with no QNN runtime, performance, or parity claim.

The PR must remain Draft with model-scale-by-skill retained.