docs: local-LLM + tool-calling feasibility findings#104
Open
chaxus wants to merge 2 commits into
Open
Conversation
Captures a recurring decision: pi/langchain are orchestration layers, not model producers/trainers; a small in-browser model's weak tool-calling is a capability ceiling, not an orchestration problem. Documents the two senses of "tuning" (inference-time vs fine-tuning), why pi != local inference, what the project already encodes (agent-core, ollama.ts native tools, webllm.ts chat-only fallback), and the ranked options: Ollama (local + capable), WebLLM + constrained decoding (zero-install lever), fine-tune (heavy), pi/langchain (wrong layer).
Deploying document with
|
| Latest commit: |
ff2a024
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a94a2fb8.document-7hm.pages.dev |
| Branch Preview URL: | https://docs-agent-llm-tradeoffs.document-7hm.pages.dev |
Expands the fine-tune option: correct ordering (cheap levers first, fine-tune only after they hit a wall), the distillation workflow (teacher model generates tool-call trajectories -> SFT/LoRA -> compile to MLC/WebLLM), the irony that bootstrapping data still needs a cloud model, ongoing retrain-on-tool-change cost, realistic expectations (a tuned 3B is still a 3B), and a go/no-go checklist for when it is actually worth it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
把反复被问到的"本地无-key agent + 工具调用"的定论写成文档,避免下次再从头讨论。
要点:
agent-core已是 pi 的等价物;webllm.ts已固化"小模型不可靠→chat-only 兜底";ollama.ts是本地+能调工具的路径详见
docs/explorations/2026-07-07-agent-llm-local-tooling-tradeoffs.md。