LLM stable aliases#340
Conversation
|
I understand the logic behind this change, however a thing to note is that this makes it even more difficult to sync with other Galaxies deploying llm_hub (e.g. .eu) as the same models are named differently. |
|
Getting the point of desired synchronization with usegalaxy.eu, however, the issue of unavailable outdated models is there too yet:
On the contrary, the model aliases in the list of this PR are claimed (by Lukas) to be at least semi-stable. Shouldn't we talk to .eu guys to adopt theese too? Obviously, I'm in favor of improving the descriptions if you have an idea. |
I will push this forward. The aliases seem like a solid solution. |
Users choose concrete service instance right now, don't they? E.g. the tool will not reroute between GPT-OSS-120B at e-Infra and GPT-OSS-120B at Uni Freiburg automatically... IMO, each Galaxy should refresh endpoints available to them independently and fairly dynamically (using cron job or dynamic tool options even :-) ). In order to make old jobs and worflows rerunable despite external service drift, the tool could fallback to the most similar model available if the original model is not available anymore and user allows it. |
Avoid querying the list of available models on ansible playbook run -- the list gets updated quite quickly and it confuses users. Instead, use a list of stable aliases described in docs: https://docs.cerit.io/en/docs/ai-as-a-service/chat-ai#model-aliases
Running the tool is not fully reproducible anymore as the underlying models can change. On the other hand, it would not be reproducible either because the obsolete models become unavailable anyway.