feat: add llmman as an LLM provider - #13207
Open
ericcurtin wants to merge 1 commit into
Open
Conversation
llmman (https://github.com/llmmanorg/llmman) is a local model runner that serves the Ollama API, so the provider subclasses the existing Ollama implementation and changes only the provider name and the default port it listens on. Model listing, FIM, embeddings and streaming all come along unchanged. Tool support delegates to the same name-based heuristic, since llmman serves the same local models. fetchModels needs no case of its own: the default branch goes through listModels, which the inherited implementation already answers from the running server rather than a remote catalog. Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Contributor
|
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds llmman, a local model runner that serves the Ollama API (alongside OpenAI- and Anthropic-compatible ones) on port 17434.
Because the wire protocol is the same, the provider subclasses the existing Ollama implementation and overrides only the provider name and default
apiBase— 18 lines instead of duplicating 833. Model listing, FIM, embeddings and streaming all carry over.Two other touchpoints:
toolSupportdelegates to the same name-based heuristic, since llmman serves the same local models.fetchModelsneeds no case of its own — thedefaultbranch goes throughlistModels, which the inherited implementation answers from the running server rather than scraping a remote catalog. That's the correct behaviour here, not an omission.Testing:
tsc --noEmit -p core/tsconfig.jsonreports 157 errors both before and after this change (all pre-existing "cannot find module" for unbuilt workspace packages like@continuedev/config-yaml), so nothing new is introduced. I couldn't run the extension end-to-end against a live server, so that part is unverified.