The main option is local llama.cpp with an OpenAI-compatible API. The same protocol works with cloud endpoints.
In General:
- Base URL - for llama.cpp the default is
http://127.0.0.1:8080. For the cloud, use the provider URL (for examplehttps://api.openai.com). - API key - stored in VSCode
SecretStorage(not in settings.json). For local llama.cpp it is usually not needed - leave it empty. - Model - pick from the list (loaded from the URL) or type it manually (model name/alias on the server).
- If needed, set the key header and scheme (defaults:
Authorization+Bearer).
Save the settings. If URL/model are missing, chat shows a hint and a button to open settings.
- Ask mode - normal chat; selected editor code is added to context.
- Agent mode - the model can call tools (read/edit files, and so on).
Switch mode with the select next to the input (or slash /ask / /agent / ...); default is Settings -> Chat.
Gen: Initialize project (.gen) or chat /init creates .gen/ (agents/commands/skills, ...), seeds .genignore for path/folder ignore, and a starter .gen/config.json for command/redact policy. Not required to chat or index - see security.md and codebase-index.md.
- In Ask, ask something simple - you should get a streaming reply (if the server supports stream).
- In Agent, ask to “read README” - a
read_filetool-call should appear.
Next: Chat and agent, Settings, Security.