How to log probs?

[XIA-Jinyi]

I want to log top-k probabilities when using the API of completion, but I can't find any introductions to relevant APIs.

[octocat20]

For Chat Completions, pass logprobs=True and optionally top_logprobs:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hi"}],
    logprobs=True,
    top_logprobs=5,
)

print(response.choices[0].logprobs)

Note: not every model/endpoint supports logprobs. If you are on the older Completions API, the param is still logprobs / top_logprobs there as well.