Skip to content

LLM stable aliases#340

Open
ljocha wants to merge 2 commits into
mainfrom
llm_stable_aliases
Open

LLM stable aliases#340
ljocha wants to merge 2 commits into
mainfrom
llm_stable_aliases

Conversation

@ljocha

@ljocha ljocha commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Avoid querying the list of available models on ansible playbook run -- the list gets updated quite quickly and it confuses users. Instead, use a list of stable aliases described in docs: https://docs.cerit.io/en/docs/ai-as-a-service/chat-ai#model-aliases

Running the tool is not fully reproducible anymore as the underlying models can change. On the other hand, it would not be reproducible either because the obsolete models become unavailable anyway.

@martenson martenson self-requested a review July 8, 2026 08:13
@martenson

martenson commented Jul 8, 2026

Copy link
Copy Markdown
Member

I understand the logic behind this change, however a thing to note is that this makes it even more difficult to sync with other Galaxies deploying llm_hub (e.g. .eu) as the same models are named differently.

@ljocha

ljocha commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Getting the point of desired synchronization with usegalaxy.eu, however, the issue of unavailable outdated models is there too yet:

openai.BadRequestError: Error code: 400 - {'error': {'message': '/chat/completions: Invalid model name passed in model=qwen3-coder-30b. Call /v1/modelsto view available models for your key.', 'type': 'None', 'param': 'None', 'code': '400', 'provider_specific_fields': {'error': '/chat/completions: Invalid model name passed in model=qwen3-coder-30b. Call/v1/models to view available models for your key.'}}}

On the contrary, the model aliases in the list of this PR are claimed (by Lukas) to be at least semi-stable. Shouldn't we talk to .eu guys to adopt theese too?

Obviously, I'm in favor of improving the descriptions if you have an idea.

@martenson

Copy link
Copy Markdown
Member

Shouldn't we talk to .eu guys to adopt theese too?

I will push this forward. The aliases seem like a solid solution.

@zsalvet

zsalvet commented Jul 8, 2026

Copy link
Copy Markdown

I understand the logic behind this change, however a thing to note is that this makes it even more difficult to sync with other Galaxies deploying llm_hub (e.g. .eu) as the same models are named differently.

Users choose concrete service instance right now, don't they? E.g. the tool will not reroute between GPT-OSS-120B at e-Infra and GPT-OSS-120B at Uni Freiburg automatically... IMO, each Galaxy should refresh endpoints available to them independently and fairly dynamically (using cron job or dynamic tool options even :-) ).

In order to make old jobs and worflows rerunable despite external service drift, the tool could fallback to the most similar model available if the original model is not available anymore and user allows it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants