[Bug]: Skywork Reward Model series not supported for `llm.reward`

[SnowCharmQ]

Your current environment

The output of python collect_env.py

==============================
        System Info
==============================
OS                           : Ubuntu 24.04.2 LTS (x86_64)
GCC version                  : (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Clang version                : Could not collect
CMake version                : Could not collect
Libc version                 : glibc-2.39

==============================
       PyTorch Info
==============================
PyTorch version              : 2.9.0+cu128
Is debug build               : False
CUDA used to build PyTorch   : 12.8
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:16:04) [GCC 11.2.0] (64-bit runtime)
Python platform              : Linux-5.10.134-19.100.al8.x86_64-x86_64-with-glibc2.39

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : True
CUDA runtime version         : Could not collect
CUDA_MODULE_LOADING set to   : 
GPU models and configuration : 
GPU 0: NVIDIA L20Y
GPU 1: NVIDIA L20Y
GPU 2: NVIDIA L20Y
GPU 3: NVIDIA L20Y
GPU 4: NVIDIA L20Y
GPU 5: NVIDIA L20Y
GPU 6: NVIDIA L20Y
GPU 7: NVIDIA L20Y

Nvidia driver version        : 570.148.08
cuDNN version                : Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_graph.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_heuristic.so.9.10.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops.so.9.10.2
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      52 bits physical, 57 bits virtual
Byte Order:                         Little Endian
CPU(s):                             192
On-line CPU(s) list:                0-191
Vendor ID:                          GenuineIntel
BIOS Vendor ID:                     Intel(R) Corporation
Model name:                         Intel(R) Xeon(R) Platinum 8468V
BIOS Model name:                    Intel(R) Xeon(R) Platinum 8468V  CPU @ 2.4GHz
BIOS CPU family:                    179
CPU family:                         6
Model:                              143
Thread(s) per core:                 2
Core(s) per socket:                 48
Socket(s):                          2
Stepping:                           8
CPU(s) scaling MHz:                 68%
CPU max MHz:                        3800.0000
CPU min MHz:                        800.0000
BogoMIPS:                           4800.00
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req hfi avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm uintr md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities
Virtualization:                     VT-x
L1d cache:                          4.5 MiB (96 instances)
L1i cache:                          3 MiB (96 instances)
L2 cache:                           192 MiB (96 instances)
L3 cache:                           195 MiB (2 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-47,96-143
NUMA node1 CPU(s):                  48-95,144-191
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced / Automatic IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

==============================
Versions of relevant libraries
==============================
[pip3] flashinfer-python==0.5.2
[pip3] numpy==2.2.6
[pip3] nvidia-cublas-cu12==12.8.4.1
[pip3] nvidia-cuda-cupti-cu12==12.8.90
[pip3] nvidia-cuda-nvrtc-cu12==12.8.93
[pip3] nvidia-cuda-runtime-cu12==12.8.90
[pip3] nvidia-cudnn-cu12==9.10.2.21
[pip3] nvidia-cudnn-frontend==1.16.0
[pip3] nvidia-cufft-cu12==11.3.3.83
[pip3] nvidia-cufile-cu12==1.13.1.3
[pip3] nvidia-curand-cu12==10.3.9.90
[pip3] nvidia-cusolver-cu12==11.7.3.90
[pip3] nvidia-cusparse-cu12==12.5.8.93
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-cutlass-dsl==4.3.2
[pip3] nvidia-ml-py==13.580.82
[pip3] nvidia-nccl-cu12==2.27.5
[pip3] nvidia-nvjitlink-cu12==12.8.93
[pip3] nvidia-nvshmem-cu12==3.3.20
[pip3] nvidia-nvtx-cu12==12.8.90
[pip3] pyzmq==27.1.0
[pip3] torch==2.9.0
[pip3] torchaudio==2.9.0
[pip3] torchvision==0.24.0
[pip3] transformers==4.57.3
[pip3] triton==3.5.0
[conda] flashinfer-python                    0.5.2            pypi_0           pypi
[conda] numpy                                2.2.6            pypi_0           pypi
[conda] nvidia-cublas-cu12                   12.8.4.1         pypi_0           pypi
[conda] nvidia-cuda-cupti-cu12               12.8.90          pypi_0           pypi
[conda] nvidia-cuda-nvrtc-cu12               12.8.93          pypi_0           pypi
[conda] nvidia-cuda-runtime-cu12             12.8.90          pypi_0           pypi
[conda] nvidia-cudnn-cu12                    9.10.2.21        pypi_0           pypi
[conda] nvidia-cudnn-frontend                1.16.0           pypi_0           pypi
[conda] nvidia-cufft-cu12                    11.3.3.83        pypi_0           pypi
[conda] nvidia-cufile-cu12                   1.13.1.3         pypi_0           pypi
[conda] nvidia-curand-cu12                   10.3.9.90        pypi_0           pypi
[conda] nvidia-cusolver-cu12                 11.7.3.90        pypi_0           pypi
[conda] nvidia-cusparse-cu12                 12.5.8.93        pypi_0           pypi
[conda] nvidia-cusparselt-cu12               0.7.1            pypi_0           pypi
[conda] nvidia-cutlass-dsl                   4.3.2            pypi_0           pypi
[conda] nvidia-ml-py                         13.580.82        pypi_0           pypi
[conda] nvidia-nccl-cu12                     2.27.5           pypi_0           pypi
[conda] nvidia-nvjitlink-cu12                12.8.93          pypi_0           pypi
[conda] nvidia-nvshmem-cu12                  3.3.20           pypi_0           pypi
[conda] nvidia-nvtx-cu12                     12.8.90          pypi_0           pypi
[conda] pyzmq                                27.1.0           pypi_0           pypi
[conda] torch                                2.9.0            pypi_0           pypi
[conda] torchaudio                           2.9.0            pypi_0           pypi
[conda] torchvision                          0.24.0           pypi_0           pypi
[conda] transformers                         4.57.3           pypi_0           pypi
[conda] triton                               3.5.0            pypi_0           pypi

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.11.2
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled
GPU Topology:
        GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    NIC0    NIC1    NIC2    NIC3    NIC4    NIC5    NIC6    NIC7    CPU AffinityNUMA Affinity   GPU NUMA ID
GPU0     X      NV8     NV8     NV8     NV8     NV8     NV8     NV8     PIX     NODE    NODE    NODE    SYS     SYS     SYS     SYS     0-47,96-1430N/A
GPU1    NV8      X      NV8     NV8     NV8     NV8     NV8     NV8     NODE    PIX     NODE    NODE    SYS     SYS     SYS     SYS     0-47,96-1430N/A
GPU2    NV8     NV8      X      NV8     NV8     NV8     NV8     NV8     NODE    NODE    PIX     NODE    SYS     SYS     SYS     SYS     0-47,96-1430N/A
GPU3    NV8     NV8     NV8      X      NV8     NV8     NV8     NV8     NODE    NODE    NODE    PIX     SYS     SYS     SYS     SYS     0-47,96-1430N/A
GPU4    NV8     NV8     NV8     NV8      X      NV8     NV8     NV8     SYS     SYS     SYS     SYS     PIX     NODE    NODE    NODE    48-95,144-191       1               N/A
GPU5    NV8     NV8     NV8     NV8     NV8      X      NV8     NV8     SYS     SYS     SYS     SYS     NODE    PIX     NODE    NODE    48-95,144-191       1               N/A
GPU6    NV8     NV8     NV8     NV8     NV8     NV8      X      NV8     SYS     SYS     SYS     SYS     NODE    NODE    PIX     NODE    48-95,144-191       1               N/A
GPU7    NV8     NV8     NV8     NV8     NV8     NV8     NV8      X      SYS     SYS     SYS     SYS     NODE    NODE    NODE    PIX     48-95,144-191       1               N/A
NIC0    PIX     NODE    NODE    NODE    SYS     SYS     SYS     SYS      X      NODE    NODE    NODE    SYS     SYS     SYS     SYS
NIC1    NODE    PIX     NODE    NODE    SYS     SYS     SYS     SYS     NODE     X      NODE    NODE    SYS     SYS     SYS     SYS
NIC2    NODE    NODE    PIX     NODE    SYS     SYS     SYS     SYS     NODE    NODE     X      NODE    SYS     SYS     SYS     SYS
NIC3    NODE    NODE    NODE    PIX     SYS     SYS     SYS     SYS     NODE    NODE    NODE     X      SYS     SYS     SYS     SYS
NIC4    SYS     SYS     SYS     SYS     PIX     NODE    NODE    NODE    SYS     SYS     SYS     SYS      X      NODE    NODE    NODE
NIC5    SYS     SYS     SYS     SYS     NODE    PIX     NODE    NODE    SYS     SYS     SYS     SYS     NODE     X      NODE    NODE
NIC6    SYS     SYS     SYS     SYS     NODE    NODE    PIX     NODE    SYS     SYS     SYS     SYS     NODE    NODE     X      NODE
NIC7    SYS     SYS     SYS     SYS     NODE    NODE    NODE    PIX     SYS     SYS     SYS     SYS     NODE    NODE    NODE     X 

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_bond_0
  NIC1: mlx5_bond_1
  NIC2: mlx5_bond_2
  NIC3: mlx5_bond_3
  NIC4: mlx5_bond_4
  NIC5: mlx5_bond_5
  NIC6: mlx5_bond_6
  NIC7: mlx5_bond_7

==============================
     Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1

🐛 Describe the bug

Following

vllm/examples/offline_inference/basic/reward.py

Lines 1 to 53 in 58d5b3f

	# SPDX-License-Identifier: Apache-2.0
	# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
	from argparse import Namespace
	from vllm import LLM, EngineArgs
	from vllm.utils.argparse_utils import FlexibleArgumentParser
	def parse_args():
	parser = FlexibleArgumentParser()
	parser = EngineArgs.add_cli_args(parser)
	# Set example specific arguments
	parser.set_defaults(
	model="internlm/internlm2-1_8b-reward",
	runner="pooling",
	enforce_eager=True,
	max_model_len=1024,
	trust_remote_code=True,
	)
	return parser.parse_args()
	def main(args: Namespace):
	# Sample prompts.
	prompts = [
	"Hello, my name is",
	"The president of the United States is",
	"The capital of France is",
	"The future of AI is",
	]
	# Create an LLM.
	# You should pass runner="pooling" for reward models
	llm = LLM(**vars(args))
	# Generate rewards. The output is a list of PoolingRequestOutput.
	outputs = llm.reward(prompts)
	# Print the outputs.
	print("\nGenerated Outputs:\n" + "-" * 60)
	for prompt, output in zip(prompts, outputs):
	rewards = output.outputs.data
	rewards_trimmed = (
	(str(rewards[:16])[:-1] + ", ...]") if len(rewards) > 16 else rewards
	)
	print(f"Prompt: {prompt!r} \nReward: {rewards_trimmed} (size={len(rewards)})")
	print("-" * 60)
	if __name__ == "__main__":
	args = parse_args()
	main(args)

, I tested Skywork/Skywork-Reward-V2-Qwen3-0.6B and the series of Skywork Reward Model, but they didn't work with the vllm==0.11.2. I got the following error:

INFO 12-09 14:32:57 [llm.py:352] Supported tasks: ['score', 'classify']
Traceback (most recent call last):
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/pgen-baselines/amazon/skywork/demo.py", line 53, in <module>
    main(args)
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/pgen-baselines/amazon/skywork/demo.py", line 38, in main
    outputs = llm.reward(prompts)
              ^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/non/lib/python3.12/site-packages/vllm/entrypoints/llm.py", line 1239, in reward
    return self.encode(
           ^^^^^^^^^^^^
  File "/root/miniconda3/envs/non/lib/python3.12/site-packages/vllm/entrypoints/llm.py", line 1073, in encode
    raise ValueError(f"pooling_task must be one of {self.supported_tasks}.")
ValueError: pooling_task must be one of ['score', 'classify'].

It seems the series of the Skywork Reward Model wasn't supported yet. Even tried with #12791 (comment), I got the error. Is there a way to support such a series of reward models? And how could them be supported for online serving?

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[noooop]

try:

llm.encode(..., pooling_task='classify')

[SnowCharmQ]

try:

llm.encode(..., pooling_task='classify')

Hi, this works. However, the output of this impl is normalized to 0-1. This will hinder the effectiveness of the reward model, I think. Is there a way to make the reward model output correctly?

The example is here:

from argparse import Namespace
from transformers import AutoTokenizer
from vllm import LLM, EngineArgs
from vllm.utils.argparse_utils import FlexibleArgumentParser


MODEL_NAME = "Skywork/Skywork-Reward-V2-Qwen3-0.6B"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

def parse_args():
    parser = FlexibleArgumentParser()
    parser = EngineArgs.add_cli_args(parser)
    parser.set_defaults(
        model=MODEL_NAME,
        runner="pooling",
        enforce_eager=True,
        max_model_len=1024,
        trust_remote_code=True,
    )
    return parser.parse_args()


prompt = "Jane has 12 apples. She gives 4 apples to her friend Mark, then buys 1 more apple, and finally splits all her apples equally among herself and her 2 siblings. How many apples does each person get?"
response1 = """1. Jane starts with 12 apples and gives 4 to Mark. 12 - 4 = 8. Jane now has 8 apples.
2. Jane buys 1 more apple. 8 + 1 = 9. Jane now has 9 apples.
3. Jane splits the 9 apples equally among herself and her 2 siblings (3 people in total). 9 ÷ 3 = 3 apples each. Each person gets 3 apples."""
response2 = """1. Jane starts with 12 apples and gives 4 to Mark. 12 - 4 = 8. Jane now has 8 apples.
2. Jane buys 1 more apple. 8 + 1 = 9. Jane now has 9 apples.
3. Jane splits the 9 apples equally among her 2 siblings (2 people in total). 9 ÷ 2 = 4.5 apples each. Each person gets 4 apples."""

conv1 = [{"role": "user", "content": prompt}, {"role": "assistant", "content": response1}]
conv2 = [{"role": "user", "content": prompt}, {"role": "assistant", "content": response2}]

conv1_formatted = tokenizer.apply_chat_template(conv1, tokenize=False)
conv2_formatted = tokenizer.apply_chat_template(conv2, tokenize=False)
if tokenizer.bos_token is not None and conv1_formatted.startswith(tokenizer.bos_token):
    conv1_formatted = conv1_formatted[len(tokenizer.bos_token):]
if tokenizer.bos_token is not None and conv2_formatted.startswith(tokenizer.bos_token):
    conv2_formatted = conv2_formatted[len(tokenizer.bos_token):]

def main(args: Namespace):
    prompts = [conv1_formatted, conv2_formatted]

    llm = LLM(**vars(args))

    outputs = llm.encode(prompts, pooling_task='classify')

    rewards = [output.outputs.data[0].item() for output in outputs]
    print(rewards)
    

if __name__ == "__main__":
    args = parse_args()
    main(args)

# Output: [0.999607264995575, 0.7310321927070618]

[noooop]

try

llm.encode(..., pooling_params=PoolingParams(activation=False), pooling_task='classify') 

or

parser.set_defaults(
    model=MODEL_NAME,
    runner="pooling",
    enforce_eager=True,
    max_model_len=1024,
    trust_remote_code=True,
    pooler_config=PoolerConfig(activation=False)
)

[SnowCharmQ]

try

llm.encode(..., pooling_params=PoolingParams(activation=False), pooling_task='classify') 

or

parser.set_defaults(
    model=MODEL_NAME,
    runner="pooling",
    enforce_eager=True,
    max_model_len=1024,
    trust_remote_code=True,
    pooler_config=PoolerConfig(activation=False)
)

out1 = llm.encode(
    {"prompt_token_ids": conv1_tokenized['input_ids']}, 
    pooling_params=PoolingParams(activation=False),
    pooling_task='classify'
)
out2 = llm.encode({"prompt_token_ids": conv2_tokenized['input_ids']}, pooling_params=PoolingParams(activation=False), pooling_task='classify')

Hi, I tried it, but got totally the same results normalized to 0-1:

conv1 tensor([1.0000])
conv2 tensor([7.5304e-05])

[noooop]

try PoolingParams(use_activation=False)

[SnowCharmQ]

try PoolingParams(use_activation=False)

Hi, I tried this, and it really works!
The results I got now are:

#conv1 tensor([11.8307])
#conv2 tensor([-9.4939])

It is very close to the hf implementation, which got the results:

# Score for response 1: 13.75
# Score for response 2: -9.375

The deviations seem to be acceptable now :)

My total implementation is attached for other followers:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "Skywork/Skywork-Reward-Llama-3.1-8B-v0.2"
rm_tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "Jane has 12 apples. She gives 4 apples to her friend Mark, then buys 1 more apple, and finally splits all her apples equally among herself and her 2 siblings. How many apples does each person get?"
response1 = """1. Jane starts with 12 apples and gives 4 to Mark. 12 - 4 = 8. Jane now has 8 apples.
2. Jane buys 1 more apple. 8 + 1 = 9. Jane now has 9 apples.
3. Jane splits the 9 apples equally among herself and her 2 siblings (3 people in total). 9 ÷ 3 = 3 apples each. Each person gets 3 apples."""
response2 = """1. Jane starts with 12 apples and gives 4 to Mark. 12 - 4 = 8. Jane now has 8 apples.
2. Jane buys 1 more apple. 8 + 1 = 9. Jane now has 9 apples.
3. Jane splits the 9 apples equally among her 2 siblings (2 people in total). 9 ÷ 2 = 4.5 apples each. Each person gets 4 apples."""

conv1 = [{"role": "user", "content": prompt}, {"role": "assistant", "content": response1}]
conv2 = [{"role": "user", "content": prompt}, {"role": "assistant", "content": response2}]

# Format and tokenize the conversations
conv1_formatted = rm_tokenizer.apply_chat_template(conv1, tokenize=False)
conv2_formatted = rm_tokenizer.apply_chat_template(conv2, tokenize=False)

conv1_tokenized = rm_tokenizer(conv1_formatted)
conv2_tokenized = rm_tokenizer(conv2_formatted)

from vllm import LLM, PoolingParams
llm = LLM(
    model=model_name, 
    task="classify",
    runner="pooling",
    override_pooler_config={"softmax": False},
    enforce_eager=True,
)
out1 = llm.encode(
    {"prompt_token_ids": conv1_tokenized['input_ids']}, 
    pooling_params=PoolingParams(use_activation=False),
    pooling_task='classify'
)
out2 = llm.encode({"prompt_token_ids": conv2_tokenized['input_ids']}, pooling_params=PoolingParams(use_activation=False), pooling_task='classify')
print('conv1', out1[0].outputs.data)
print('conv2', out2[0].outputs.data)
#conv1 tensor([11.8307])
#conv2 tensor([-9.4939])

[SnowCharmQ]

try PoolingParams(use_activation=False)

Btw, for online serving, I just need to pass the required parameters the same as offline serving here right?

[noooop]

try --pooler-config='{"use_ activation": false}' https://docs.vllm.ai/en/latest/cli/serve/?h=pooler#-pooler-config

[SnowCharmQ]

try --pooler-config='{"use_ activation": false}' https://docs.vllm.ai/en/latest/cli/serve/?h=pooler#-pooler-config

Hi,
thank you for your previous help with offline serving. But I still encounter issues with online serving.

The following is my bash script to start the online server:

#!/bin/bash

export CUDA_VISIBLE_DEVICES=7

LOG_FILE="server_$(date +%Y%m%d_%H%M%S).log"

vllm serve Skywork/Skywork-Reward-V2-Qwen3-0.6B \
    --seed 42 \
    --tensor-parallel-size 1 \
    --dtype bfloat16 \
    --api-key EMPTY \
    --max-model-len 8192 \
    --max-num-batched-tokens 8192 \
    --gpu-memory-utilization 0.9 \
    --enable-chunked-prefill \
    --task classify \
    --runner pooling \
    --pooler-config '{"softmax": false}' \
    2>&1 | tee ${LOG_FILE}

The server looks good, however when I tried to request it similar to offline serving I encounter the issue:

{'error': {'code': 400,
           'message': '3 validation errors:\n'
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'list[str]', 0), 'msg': 'Input should be a valid "
                      "string', 'input': {'prompt_token_ids': [9707, 11, 847, "
                      '829, 374]}}\n'
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'str'), 'msg': 'Input should be a valid string', "
                      "'input': [{'prompt_token_ids': [9707, 11, 847, 829, "
                      '374]}]}\n'
                      "  {'type': 'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      "[{'prompt_token_ids': [9707, 11, 847, 829, 374]}]}}\n"
                      '\n'
                      '  File '
                      '"/root/miniconda3/envs/verl/lib/python3.12/site-packages/vllm/entrypoints/utils.py", '
                      'line 823, in create_classify\n'
                      "    POST /classify [{'type': 'string_type', 'loc': "
                      "('body', 'function-wrap[__log_extra_fields__()]', "
                      "'input', 'list[str]', 0), 'msg': 'Input should be a "
                      "valid string', 'input': {'prompt_token_ids': [9707, 11, "
                      "847, 829, 374]}}, {'type': 'string_type', 'loc': "
                      "('body', 'function-wrap[__log_extra_fields__()]', "
                      "'input', 'str'), 'msg': 'Input should be a valid "
                      "string', 'input': [{'prompt_token_ids': [9707, 11, 847, "
                      "829, 374]}]}, {'type': 'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      "[{'prompt_token_ids': [9707, 11, 847, 829, 374]}]}}]",
           'param': None,
           'type': 'Bad Request'}}

It seems that I have to pass prompt strings instead of tokenized inputs_ids similar to offline serving.
My client implementation is:

import argparse
import pprint

import requests
from transformers import AutoTokenizer

def post_http_request(payload: dict, api_url: str) -> requests.Response:
    headers = {"User-Agent": "Test Client"}
    response = requests.post(api_url, headers=headers, json=payload)
    return response


def parse_args():
    parse = argparse.ArgumentParser()
    parse.add_argument("--host", type=str, default="localhost")
    parse.add_argument("--port", type=int, default=8000)
    parse.add_argument("--model", type=str, default="Skywork/Skywork-Reward-V2-Qwen3-0.6B")
    return parse.parse_args()


def main(args):
    host = args.host
    port = args.port
    model_name = args.model

    api_url = f"http://{host}:{port}/classify"
    prompts = [
        "Hello, my name is",
    ]
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    tokenized_prompts = tokenizer(prompts)
    input_ids = tokenized_prompts['input_ids']
    input_ids = [{'prompt_token_ids': iids} for iids in input_ids]

    payload = {
        "model": model_name,
        "input": input_ids,
    }

    classify_response = post_http_request(payload=payload, api_url=api_url)
    pprint.pprint(classify_response.json())


if __name__ == "__main__":
    args = parse_args()
    main(args)

[noooop]

vllm/vllm/entrypoints/pooling/embed/protocol.py

Lines 18 to 26 in 434ac76

	class EmbeddingCompletionRequest(OpenAIBaseModel):
	# Ordered by official OpenAI API documentation
	# https://platform.openai.com/docs/api-reference/embeddings
	model: str | None = None
	input: list[int] | list[list[int]] | str | list[str]
	encoding_format: EncodingFormat = "float"
	dimensions: int | None = None
	user: str | None = None
	truncate_prompt_tokens: Annotated[int, Field(ge=-1)] | None = None

try

input: list[int] | list[list[int]]

without 'prompt_token_ids'

[SnowCharmQ]

vllm/vllm/entrypoints/pooling/embed/protocol.py

Tried it but got almost the same error:

{'error': {'code': 400,
           'message': '3 validation errors:\n'
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'list[str]', 0), 'msg': 'Input should be a valid "
                      "string', 'input': [9707, 11, 847, 829, 374]}\n"
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'str'), 'msg': 'Input should be a valid string', "
                      "'input': [[9707, 11, 847, 829, 374]]}\n"
                      "  {'type': 'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      '[[9707, 11, 847, 829, 374]]}}\n'
                      '\n'
                      '  File '
                      '"/root/miniconda3/envs/non/lib/python3.12/site-packages/vllm/entrypoints/utils.py", '
                      'line 823, in create_classify\n'
                      "    POST /classify [{'type': 'string_type', 'loc': "
                      "('body', 'function-wrap[__log_extra_fields__()]', "
                      "'input', 'list[str]', 0), 'msg': 'Input should be a "
                      "valid string', 'input': [9707, 11, 847, 829, 374]}, "
                      "{'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'str'), 'msg': 'Input should be a valid string', "
                      "'input': [[9707, 11, 847, 829, 374]]}, {'type': "
                      "'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      '[[9707, 11, 847, 829, 374]]}}]',
           'param': None,
           'type': 'Bad Request'}}

Only commented one line of my previous implementation:

import argparse
import pprint

import requests
from transformers import AutoTokenizer

def post_http_request(payload: dict, api_url: str) -> requests.Response:
    headers = {"User-Agent": "Test Client"}
    response = requests.post(api_url, headers=headers, json=payload)
    return response


def parse_args():
    parse = argparse.ArgumentParser()
    parse.add_argument("--host", type=str, default="localhost")
    parse.add_argument("--port", type=int, default=8000)
    parse.add_argument("--model", type=str, default="Skywork/Skywork-Reward-V2-Qwen3-0.6B")
    return parse.parse_args()


def main(args):
    host = args.host
    port = args.port
    model_name = args.model

    api_url = f"http://{host}:{port}/classify"
    prompts = [
        "Hello, my name is",
    ]
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    tokenized_prompts = tokenizer(prompts)
    input_ids = tokenized_prompts['input_ids']
    # input_ids = [{'prompt_token_ids': iids} for iids in input_ids]

    payload = {
        "model": model_name,
        "input": input_ids,
    }

    classify_response = post_http_request(payload=payload, api_url=api_url)
    pprint.pprint(classify_response.json())


if __name__ == "__main__":
    args = parse_args()
    main(args)

The vllm version is 0.11.2, I am going to try with the latest version of vllm.

[SnowCharmQ]

vllm/vllm/entrypoints/pooling/embed/protocol.py

Tried it but got almost the same error:

{'error': {'code': 400,
           'message': '3 validation errors:\n'
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'list[str]', 0), 'msg': 'Input should be a valid "
                      "string', 'input': [9707, 11, 847, 829, 374]}\n"
                      "  {'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'str'), 'msg': 'Input should be a valid string', "
                      "'input': [[9707, 11, 847, 829, 374]]}\n"
                      "  {'type': 'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      '[[9707, 11, 847, 829, 374]]}}\n'
                      '\n'
                      '  File '
                      '"/root/miniconda3/envs/non/lib/python3.12/site-packages/vllm/entrypoints/utils.py", '
                      'line 823, in create_classify\n'
                      "    POST /classify [{'type': 'string_type', 'loc': "
                      "('body', 'function-wrap[__log_extra_fields__()]', "
                      "'input', 'list[str]', 0), 'msg': 'Input should be a "
                      "valid string', 'input': [9707, 11, 847, 829, 374]}, "
                      "{'type': 'string_type', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'input', "
                      "'str'), 'msg': 'Input should be a valid string', "
                      "'input': [[9707, 11, 847, 829, 374]]}, {'type': "
                      "'missing', 'loc': ('body', "
                      "'function-wrap[__log_extra_fields__()]', 'messages'), "
                      "'msg': 'Field required', 'input': {'model': "
                      "'Skywork/Skywork-Reward-V2-Qwen3-0.6B', 'input': "
                      '[[9707, 11, 847, 829, 374]]}}]',
           'param': None,
           'type': 'Bad Request'}}

Only commented one line of my previous implementation:

import argparse
import pprint

import requests
from transformers import AutoTokenizer

def post_http_request(payload: dict, api_url: str) -> requests.Response:
headers = {"User-Agent": "Test Client"}
response = requests.post(api_url, headers=headers, json=payload)
return response

def parse_args():
parse = argparse.ArgumentParser()
parse.add_argument("--host", type=str, default="localhost")
parse.add_argument("--port", type=int, default=8000)
parse.add_argument("--model", type=str, default="Skywork/Skywork-Reward-V2-Qwen3-0.6B")
return parse.parse_args()

def main(args):
host = args.host
port = args.port
model_name = args.model

api_url = f"http://{host}:{port}/classify"
prompts = [
    "Hello, my name is",
]
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenized_prompts = tokenizer(prompts)
input_ids = tokenized_prompts['input_ids']
# input_ids = [{'prompt_token_ids': iids} for iids in input_ids]

payload = {
    "model": model_name,
    "input": input_ids,
}

classify_response = post_http_request(payload=payload, api_url=api_url)
pprint.pprint(classify_response.json())

if name == "main":
args = parse_args()
main(args)
The vllm version is 0.11.2, I am going to try with the latest version of vllm.

Just tried it with the latest built vllm and got totally the same error. I just think the reason behind the issue is, the server doesn't invoke .embed, and I didn't find the API for .reward since I had already tried it.

For your reference, the output of the server is attached:

(APIServer pid=942680) INFO 12-10 16:48:28 [api_server.py:1425] Starting vLLM API server 0 on http://0.0.0.0:8000
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:38] Available routes are:
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /openapi.json, Methods: HEAD, GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /docs, Methods: HEAD, GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /redoc, Methods: HEAD, GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /pause, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /resume, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /is_paused, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /classify, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/embeddings, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /score, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/score, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /rerank, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v1/rerank, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /v2/rerank, Methods: POST
(APIServer pid=942680) INFO 12-10 16:48:28 [launcher.py:46] Route: /pooling, Methods: POST
(APIServer pid=942680) INFO:     Started server process [942680]
(APIServer pid=942680) INFO:     Waiting for application startup.
(APIServer pid=942680) INFO:     Application startup complete.
(APIServer pid=942680) INFO:     127.0.0.1:49522 - "POST /classify HTTP/1.1" 400 Bad Request