Skip to content

_maybe_add_1m_suffix assumes [1m] model variants exist; gateway 400s when they don't #300

Description

@fflory

Summary

_maybe_add_1m_suffix infers the existence of a [1m] 1M-context model variant
from the version number alone. On a workspace whose AI Gateway does not serve
those variants, configure writes a model name the gateway rejects, and the
affected tier stops working with no actionable error.

Details

src/ucode/agents/claude.py:375-390:

should_suffix = (family == "opus" and (major, minor) >= (4, 6)) or (
    family == "sonnet" and (major, minor) >= (4, 6)
)
return f"{model}[1m]" if should_suffix else model

The decision is purely version-derived. Nothing checks whether the workspace
advertises the suffixed variant — even though /ai-gateway/anthropic/v1/models
returns an authoritative list, and on my workspace that list contains
databricks-claude-opus-4-8 but no [1m] form.

Result against that gateway (POST /ai-gateway/anthropic/v1/messages):

Model name HTTP
databricks-claude-opus-4-8[1m] 400 Invalid name
databricks-claude-opus-4-8 200
databricks-claude-sonnet-4-6[1m] 400 Invalid name
databricks-claude-sonnet-4-6 200

There is also no escape hatch: the only UCODE_* env knobs are UCODE_DEBUG,
UCODE_HOME, and UCODE_WEB_SEARCH_MODEL, and no CLI flag disables the
suffixing. If discovery selects an affected model as a family model, that tier
is unusable until the settings file is hand-edited — which configure then
overwrites.

Why it is currently latent (and why that is fragile)

The regex at claude.py:88 requires a major and minor version:

r"^(?:system\.ai\.)?(?:databricks-)?claude-(opus|sonnet)-(\d+)-(\d+)(.*)$"

So claude-opus-5 and claude-sonnet-5 — no minor component — never match, and
never get suffixed. That is the only reason my configure produced a working
config: discovery happened to pick the -5 models. Selecting a 4.6/4.7/4.8
model as a family model, on this same workspace, would have produced a broken
tier. The safety here is incidental to a naming coincidence, not by design.

Suggested fix

Gate the suffix on the model actually being served, rather than on its version:
check the /v1/models listing (or the model-services listing) for the [1m]
form and only emit it when present. Falling back to the unsuffixed name is
strictly safer than emitting a name the gateway will reject — 1M context is an
enhancement, so losing it silently is a much better failure mode than a
non-functional tier.

A --no-1m / UCODE_DISABLE_1M escape hatch would also be a cheap
independent mitigation.

Environment

  • ucode 0.0.0+174.g62c5536 (commit 62c5536, 2026-08-07)
  • Databricks CLI 1.10.0, macOS 15 (Darwin 25.6.0)
  • AI Gateway v2 (/ai-gateway/anthropic) on a Databricks-on-AWS workspace
  • Agent: Claude Code (native install)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions