Skip to content

feat: add llmman as an LLM provider - #13207

Open
ericcurtin wants to merge 1 commit into
continuedev:mainfrom
ericcurtin:feat/llmman-provider
Open

feat: add llmman as an LLM provider#13207
ericcurtin wants to merge 1 commit into
continuedev:mainfrom
ericcurtin:feat/llmman-provider

Conversation

@ericcurtin

Copy link
Copy Markdown

Adds llmman, a local model runner that serves the Ollama API (alongside OpenAI- and Anthropic-compatible ones) on port 17434.

Because the wire protocol is the same, the provider subclasses the existing Ollama implementation and overrides only the provider name and default apiBase — 18 lines instead of duplicating 833. Model listing, FIM, embeddings and streaming all carry over.

Two other touchpoints:

  • toolSupport delegates to the same name-based heuristic, since llmman serves the same local models.
  • fetchModels needs no case of its own — the default branch goes through listModels, which the inherited implementation answers from the running server rather than scraping a remote catalog. That's the correct behaviour here, not an omission.

Testing: tsc --noEmit -p core/tsconfig.json reports 157 errors both before and after this change (all pre-existing "cannot find module" for unbuilt workspace packages like @continuedev/config-yaml), so nothing new is introduced. I couldn't run the extension end-to-end against a live server, so that part is unverified.

AI-assisted, reviewed before submitting.

llmman (https://github.com/llmmanorg/llmman) is a local model runner that
serves the Ollama API, so the provider subclasses the existing Ollama
implementation and changes only the provider name and the default port it
listens on. Model listing, FIM, embeddings and streaming all come along
unchanged.

Tool support delegates to the same name-based heuristic, since llmman
serves the same local models.

fetchModels needs no case of its own: the default branch goes through
listModels, which the inherited implementation already answers from the
running server rather than a remote catalog.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@ericcurtin
ericcurtin requested a review from a team as a code owner August 30, 2026 23:55
@ericcurtin
ericcurtin requested review from sestinj and removed request for a team August 30, 2026 23:55
@github-actions

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant