Skip to content

[codex] fix(ollama): honor request option overrides#12022

Open
yzlu0917 wants to merge 1 commit intocontinuedev:mainfrom
yzlu0917:codex/issue-10378-ollama-autocomplete-options
Open

[codex] fix(ollama): honor request option overrides#12022
yzlu0917 wants to merge 1 commit intocontinuedev:mainfrom
yzlu0917:codex/issue-10378-ollama-autocomplete-options

Conversation

@yzlu0917
Copy link
Copy Markdown

@yzlu0917 yzlu0917 commented Apr 3, 2026

Summary

  • add first-class config support for Ollama request overrides via requestOptions.keepAlive and requestOptions.options
  • merge those request-level overrides into Ollama /api/generate and /api/chat bodies for both autocomplete and chat requests
  • add regression tests covering config parsing and Ollama request-body merging, while preserving completion-option precedence

Why

Continue already supports Ollama-specific runtime controls like keep_alive, num_gpu, and num_thread, but the practical config path users were relying on (requestOptions.options) was not actually making it through YAML parsing or into the Ollama request body. As a result, Continue could silently drop model residency and hardware-allocation settings, causing Ollama to reload models with default parameters and evict other preloaded models.

This change makes those request overrides survive config parsing and flow all the way into Ollama chat and generate requests.

Validation

  • ran npm test -- --runInBand llm/llms/Ollama.test.ts in core
  • ran npm test -- --runInBand src/schemas/models.test.ts in packages/config-yaml
  • ran npm run build in packages/config-yaml
  • ran npm run build in packages/config-types
  • ran git diff --check
  • ran npm run tsc:check in core and only hit the existing unrelated OPENROUTER_HEADERS export mismatch in llm/llms/OpenRouter.ts

Closes #10378


Summary by cubic

Honors Ollama request option overrides from config by parsing requestOptions.keepAlive and requestOptions.options and merging them into /api/generate and /api/chat. Prevents dropped model residency and hardware settings, and keeps completion options as the source of truth when provided.

  • Bug Fixes
    • Parse and expose keepAlive and options in requestOptions in core, packages/config-yaml, and packages/config-types.
    • Merge these overrides into both autocomplete and chat requests via a shared base options builder.
    • Keep completion options (e.g., keepAlive, numGpu, numThreads) overriding request-level defaults.
    • Add tests for YAML schema parsing and request-body merging to prevent regressions.

Written for commit 6ac093b. Summary will update on new commits.

@yzlu0917 yzlu0917 requested a review from a team as a code owner April 3, 2026 08:23
@yzlu0917 yzlu0917 requested review from sestinj and removed request for a team April 3, 2026 08:23
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Apr 3, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ac093b5d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

stream: options.stream,
// format: options.format, // Not currently in base completion options
};
if (options.tools?.length && ollamaMessages.at(-1)?.role === "user") {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reinstate Ollama tool-support guard before attaching tools

The new condition now always adds chatOptions.tools for user turns, but the prior templateSupportsTools !== false safeguard was removed. That means models whose /api/show template does not include .Tools will still receive tool definitions and can reject the request, causing tool-enabled chats to fail instead of degrading gracefully. This regression is user-visible whenever Ollama is configured with a model lacking tool-template support.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Ollama autocomplete model unloads chat model despite keep_alive: -1 and num_gpu settings

1 participant