diff --git a/docs/toolhive/concepts/tool-optimization.mdx b/docs/toolhive/concepts/tool-optimization.mdx index 008c5707..8a1e53b1 100644 --- a/docs/toolhive/concepts/tool-optimization.mdx +++ b/docs/toolhive/concepts/tool-optimization.mdx @@ -252,7 +252,7 @@ context-specific information that helps the AI understand when to use each tool. Now that you understand when and why to use tool filtering and overrides, learn how to configure them: -- [Customize tools (CLI)](../guides-cli/run-mcp-servers.mdx) +- [Customize tools (CLI)](../guides-cli/configure-mcp-servers.mdx#run-a-server-exposing-only-selected-tools) - [Customize tools (UI)](../guides-ui/customize-tools.mdx) - [Customize tools (Kubernetes)](../guides-k8s/customize-tools.mdx) - [MCPToolConfig CRD reference](../reference/crds/index.mdx) diff --git a/docs/toolhive/guides-cli/configure-mcp-servers.mdx b/docs/toolhive/guides-cli/configure-mcp-servers.mdx new file mode 100644 index 00000000..efa33907 --- /dev/null +++ b/docs/toolhive/guides-cli/configure-mcp-servers.mdx @@ -0,0 +1,429 @@ +--- +title: Configure MCP servers +description: + Customize MCP server behavior with the ToolHive CLI, including names, secrets, + ports, tool filtering, audit logging, volumes, and network isolation. +--- + +You might need to customize the behavior of an MCP server, such as changing the +port, mounting a local directory, or passing secrets. ToolHive provides several +options to customize a server's configuration when you run it. + +These options apply whether you run a server from the +[registry](./run-mcp-servers.mdx#run-a-server-from-the-registry), from a +[custom image or protocol scheme](./run-mcp-servers.mdx#run-a-custom-mcp-server), +or as a [remote server](./run-remote-mcp-servers.mdx) - but not all options +apply to every mode. They fall into two groups: + +- [**Options for all servers**](#options-for-all-servers) operate at the proxy + or workload level, so they work for registry, custom, and remote servers. +- [**Options for local servers**](#options-for-local-servers) configure the + container that ToolHive runs on your machine. They have no effect on remote + servers, which proxy to a URL instead of running a local container. + +For a complete list of options, run +[`thv run --help`](../reference/cli/thv_run.md) or see the +[`thv run` command reference](../reference/cli/thv_run.md). + +## Options for all servers + +These options apply to every run mode, including remote servers. + +### Run a server with a custom name + +By default, the workload name matches the MCP server's name in the registry or +is automatically generated from the image name when you run a custom server. To +give your server instance a custom name, use the `--name` option: + +```bash +thv run --name +``` + +For example: + +```bash +thv run --name my-fetch fetch +``` + +### Run a server within a group + +To run an MCP server within a specific group, use the `--group` option. This +allows you to organize your servers and manage them collectively. + +```bash +thv run --group +``` + +:::note + +The group must exist before you can run a server in it. + +::: + +See [Organize servers into groups](./group-management.mdx) to learn more about +organizing servers and configuring client access. + +### Run a server on a specific port + +ToolHive creates a reverse proxy on a random port that forwards requests to the +server. This is the port that client applications connect to. To set a specific +proxy port instead, use the `--proxy-port` flag: + +```bash +thv run --proxy-port +``` + +For local servers, you can also publish the container's own port directly to the +host (separate from the proxy port) with `--publish` (`-p`) using +`hostPort:containerPort`. Repeat the flag for multiple ports: + +```bash +thv run --publish 8090:8080 +``` + +Most servers only need the proxy port; use `--publish` when a container exposes +an additional port you need to reach directly. + +### Override the session timeout + +ToolHive's proxy evicts idle MCP sessions after 2 hours by default. To raise or +lower this inactivity timeout for a workload, pass `--session-ttl` with a Go +duration string: + +```bash +thv run --session-ttl 4h +``` + +Set a longer value when clients hold sessions open for long-running operations, +or a shorter value to free resources faster. + +### Run a server in the foreground + +By default, ToolHive runs the server in the background and returns control to +your shell. To keep the process attached to your terminal until the server +stops, use the `--foreground` (`-f`) flag: + +```bash +thv run --foreground +``` + +This is useful for debugging or when you want the server's lifecycle tied to the +current shell session. + +### Run a server exposing only selected tools + +ToolHive can filter the tools returned to the client as result of a `tools/list` +command as well as block calls to tools that you don't want to expose. + +This can help reduce the amount of tools sent to the LLM while still using the +same MCP server, but it is not meant as a security feature. + +To filter the list of tools, use the `--tools` flag either once + +```bash +thv run --tools +``` + +Or multiple times + +```bash +thv run --tools --tools +``` + +For example: + +```bash +thv run --tools list_issues --tools get_issue github +``` + +If the server comes from the registry, ToolHive can validate the tool names +against the list advertised in the image reference. An error is returned in case +ToolHive cannot find one of the specified tools. + +### Override tool names and descriptions + +With ToolHive you can modify how tools exposed by an MCP server are exposed to +clients. In particular, tool names and descriptions can be changed. + +This is useful when you want to guide an agent toward calling a specific tool +for particular questions. + +One common use case is running multiple copies of the same MCP server with +different [network access levels](./network-isolation.mdx), one for internal +resources and another for public internet access. By overriding the tool names +and descriptions, you can help your agent choose the right server for each task. + +For example, the `fetch` MCP server exposes a single `fetch` tool with a +description like: + +``` +"Fetches a URL from the internet and optionally extracts its contents as markdown." +``` + +To override this, create a configuration file with one entry under +`toolsOverride` for each tool you want to modify: + +```json +{ + "toolsOverride": { + "fetch": { + "name": "toolhive-docs-fetch", + "description": "Fetches a URL from https://docs.stacklok.com/toolhive website." + } + } +} +``` + +Then pass this file to [`thv run`](../reference/cli/thv_run.md): + +```sh +thv run --tools-override override.json fetch +``` + +The key in the override object is the _original tool name_, while the `name` +field contains the _new name_ that clients will see. + +:::info + +Take care when using `--tools` and `--tools-override` together in the same +command. + +Tool filtering and tool overrides work independently: filtering limits access to +a subset of tools, while overrides change how those tools appear to clients. + +When using both options, `--tools` must reference the _overridden names_ (the +new names you define) since those are what clients will see. + +::: + +### Enable audit logging + +ToolHive can record an audit trail of the MCP activity that flows through a +server's proxy. Audit events capture interactions such as client initialization, +`tools/list` and `tools/call` requests, resource reads, prompt requests, and +notifications, along with metadata like the timestamp, component name, and +outcome. + +To turn on audit logging with ToolHive's built-in defaults, use the +`--enable-audit` flag: + +```bash +thv run --enable-audit +``` + +The default configuration records each event type but omits the request and +response payloads to protect potentially sensitive data. + +The audit middleware runs in the ToolHive proxy process, not in the MCP server +container, so audit events are written to the proxy's logs. Pass the `--proxy` +flag to `thv logs` to view them: + +```bash +thv logs --proxy +``` + +Each event is emitted as a JSON object on its own line, which makes the output +easy to filter or forward to a log aggregator. + +For finer control, supply your own configuration file with the `--audit-config` +flag instead: + +```bash +thv run --audit-config ./audit.json +``` + +A configuration file lets you select which event types to record with +`eventTypes` (or exclude specific types with `excludeEventTypes`), choose +whether to include request or response data, and set where to send the audit +log. For example: + +```json title="audit.json" +{ + "component": "fetch-server", + "eventTypes": ["mcp_tool_call", "mcp_resource_read"], + "includeRequestData": true, + "includeResponseData": false, + "maxDataSize": 2048, + "logFile": "./fetch-audit.log" +} +``` + +When you set `logFile`, ToolHive appends audit events to that file instead of +the proxy logs. Use a path that the user running `thv` can write to. Leave it +unset to keep events in the proxy logs viewed by `thv logs --proxy`. + +:::note + +Use either flag to turn on audit logging: `--enable-audit` for the built-in +defaults, or `--audit-config` to load your own settings. `--audit-config` +enables auditing on its own, so you don't need to pass `--enable-audit` as well. + +::: + +Setting `includeRequestData` or `includeResponseData` to `true` can capture +sensitive arguments or results in the audit log. Enable these options only when +you understand the privacy implications, and adjust `maxDataSize` to limit how +much of each payload is recorded. + +## Options for local servers + +These options configure the container that ToolHive runs on your machine for +registry and custom servers. They have no effect on remote servers, which proxy +to a URL instead of running a local container. + +### Pass environment variables + +Many MCP servers read non-sensitive configuration from environment variables. +Pass them individually with `-e` / `--env`: + +```bash +thv run -e = -e = +``` + +To load variables from a file instead, use `--env-file`. To load every file in a +directory, use `--env-file-dir`: + +```bash +thv run --env-file ./server.env +thv run --env-file-dir ./env.d +``` + +For sensitive values such as API tokens, use `--secret` instead, so the value is +resolved from your secrets provider rather than passed in plaintext. + +### Run a server with secrets + +Many MCP servers require secrets such as API tokens to function correctly. +ToolHive resolves these from your secrets provider and injects them into the +container as environment variables. + +To pass a secret to an MCP server, use the `--secret` option: + +```bash +thv run --secret ,target= +``` + +The `target` parameter specifies the name of the environment variable in the MCP +server's container. + +For example: + +```bash +thv run --secret github,target=GITHUB_PERSONAL_ACCESS_TOKEN github +``` + +See [Secrets management](./secrets-management.mdx) to learn how to manage +secrets in ToolHive. To supply credentials to a remote server, see +[authentication setup](./run-remote-mcp-servers.mdx#authentication-setup). + +### Mount a local file or directory + +To enable file system access for an MCP server, you can either use the +`--volume` flag to mount specific paths or create a custom permission profile +that defines read and write permissions. + +See [File system access](./filesystem-access.mdx) for detailed examples. To +prevent sensitive files from being exposed when mounting a project, use +[.thvignore](./thvignore.mdx). + +### Restrict network access + +ToolHive enables _network isolation_ by default, restricting an MCP server's +outbound traffic to the hosts and ports declared in its permission profile. +Override the default profile with `--permission-profile`, or opt out entirely +with `--isolate-network=false`. + +See [Network isolation](./network-isolation.mdx) for network architecture +details and examples. + +### Add command-line arguments + +Some MCP servers require additional arguments to run correctly. You can pass +these arguments after the server name in the +[`thv run`](../reference/cli/thv_run.md) command: + +```bash +thv run -- +``` + +For example: + +```bash +thv run my-mcp-server:latest -- --arg1 value1 --arg2 value2 +``` + +Check the MCP server's documentation for the required arguments. + +:::warning + +Some MCP servers in the ToolHive registry include default arguments that are +essential for proper operation. When you provide custom arguments using +`-- `, these replace the registry defaults entirely rather than adding to +them. + +Before adding custom arguments, check the server's registry entry: + +```bash +thv registry info --format json | jq '.args' +``` + +If default arguments are listed, include them along with your custom arguments +to ensure the server functions correctly. + +::: + +### Verify container image signatures + +For MCP servers that run from a container image, ToolHive can check the image's +signature and provenance before launching it. Some registry servers publish +provenance information that ToolHive uses to confirm the image was built and +signed as expected. To see whether a given server has it, check the +`Has Provenance` field in the output of `thv registry info `. + +Use the `--image-verification` flag to control this behavior. It accepts three +values: + +- `warn` (default): verify the image and log a warning if verification fails or + the server has no provenance information, but run the server anyway. +- `enabled`: verify the image and refuse to run the server if verification fails + or no provenance information is available. +- `disabled`: skip image verification entirely. + +```bash +thv run --image-verification enabled +``` + +For example, to enforce verification when running the `fetch` server: + +```bash +thv run --image-verification enabled fetch +``` + +:::note + +Verification succeeds only for servers that include provenance information, and +not every server does. When a server has no provenance data, `warn` runs it with +a warning while `enabled` declines to run it. If you set `enabled`, keep in mind +that it stops servers without provenance, not only servers that fail +verification. + +::: + +## Next steps + +- [Monitor and manage MCP servers](./manage-mcp-servers.mdx) to control your + running servers +- [Test your MCP servers](./test-mcp-servers.mdx) using the MCP Inspector or + `thv mcp` commands +- [Secure your servers](./auth.mdx) with OIDC authentication and Cedar policies + +## Related information + +- [`thv run` command reference](../reference/cli/thv_run.md) +- [Run MCP servers](./run-mcp-servers.mdx) +- [Run remote MCP servers](./run-remote-mcp-servers.mdx) +- [Enable telemetry and metrics](./telemetry-and-metrics.mdx) +- [Secrets management](./secrets-management.mdx) +- [Custom permissions](./custom-permissions.mdx) + - [File system access](./filesystem-access.mdx) + - [Network isolation](./network-isolation.mdx) diff --git a/docs/toolhive/guides-cli/run-mcp-servers.mdx b/docs/toolhive/guides-cli/run-mcp-servers.mdx index 96c1a6f4..0ac0fd54 100644 --- a/docs/toolhive/guides-cli/run-mcp-servers.mdx +++ b/docs/toolhive/guides-cli/run-mcp-servers.mdx @@ -1,13 +1,20 @@ --- title: Run MCP servers description: - Run MCP servers with the ToolHive CLI, including registry shortcuts, custom - images, and environment variables. + Run MCP servers with the ToolHive CLI from the registry, custom Docker images, + or protocol schemes. --- This guide explains how to run Model Context Protocol (MCP) servers using -ToolHive. It covers how to run servers from the ToolHive registry, customize -server settings, and run custom servers using Docker images or protocol schemes. +ToolHive. It covers how to run servers from the ToolHive registry and how to run +custom servers using Docker images or protocol schemes. + +Related guides cover the rest of the workflow: + +- [Run remote MCP servers](./run-remote-mcp-servers.mdx) to connect to servers + hosted elsewhere by URL, including OAuth and bearer token authentication. +- [Configure MCP servers](./configure-mcp-servers.mdx) to customize a server + with options like custom names, tool filtering, ports, and volume mounts. ## Run a server from the registry @@ -39,7 +46,7 @@ ToolHive's transparent HTTP proxy to forward requests to remote servers. For example: ```bash -thv run neon +thv run notion-remote thv run stripe ``` @@ -48,38 +55,14 @@ remote URL and authentication settings. :::note[Naming convention] -Remote MCP servers use the `-remote` suffix **when they have a local -containerized counterpart** to distinguish between the two versions. For -example: - -- `notion-remote` indicates this is the remote version of a server that also has - a local `notion` version -- `neon` and `stripe` don't have local counterparts, so they don't use the - `-remote` suffix - -To run a remote notion mcp server, you should use the `notion-remote` name. - -```bash -thv run notion-remote -``` - -Use `thv search ` or `thv registry list` to discover available servers. +Remote MCP servers use the `-remote` suffix when they have a local containerized +counterpart, to distinguish the two versions (for example, `notion-remote`). +Servers without a local counterpart, like `stripe`, don't use the suffix. ::: -### Run servers in a group - -You can organize MCP servers into groups using `thv group create` and then run -servers within a group: - -```bash -thv group create my-group -thv run --group my-group fetch -thv run --group my-group github -``` - -See [Group management](./group-management.mdx) for more details on creating and -managing groups. +To connect to a remote server that isn't in the registry, see +[Run remote MCP servers](./run-remote-mcp-servers.mdx). :::info[What's happening?] @@ -110,390 +93,27 @@ types automatically: ::: -See [Run a custom MCP server](#run-a-custom-mcp-server) to run a server that is -not in the registry, or [Run a remote MCP server](#run-a-remote-mcp-server) for -more details about remote server configuration. - -## Customize server settings - -You might need to customize the behavior of an MCP server, such as changing the -port, mounting a local directory, or passing secrets. ToolHive provides several -options to customize the server's configuration when you run it. - -For a complete list of options, run -[`thv run --help`](../reference/cli/thv_run.md) or see the -[`thv run` command reference](../reference/cli/thv_run.md). - -### Run a server with a custom name - -By default, the container name matches the MCP server's name in the registry or -is automatically generated from the image name when you run a custom server. To -give your server instance a custom name, use the `--name` option: - -```bash -thv run --name -``` - -For example: - -```bash -thv run --name my-fetch fetch -``` - -### Run a server with secrets - -Many MCP servers require secrets or other configuration variables to function -correctly. ToolHive lets you pass these secrets as environment variables when -starting the server. - -To pass a secret to an MCP server, use the `--secret` option: - -```bash -thv run --secret ,target= -``` - -The `target` parameter specifies the name of the environment variable in the MCP -server's container. This is useful for passing secrets like API tokens or other -sensitive information. - -For example: - -```bash -thv run --secret github,target=GITHUB_PERSONAL_ACCESS_TOKEN github -``` - -See [Secrets management](./secrets-management.mdx) to learn how to manage -secrets in ToolHive. - -### Pass environment variables - -Many MCP servers read non-sensitive configuration from environment variables. -Pass them individually with `-e` / `--env`: - -```bash -thv run -e = -e = -``` - -To load variables from a file instead, use `--env-file`. To load every file in a -directory, use `--env-file-dir`: - -```bash -thv run --env-file ./server.env -thv run --env-file-dir ./env.d -``` - -For sensitive values such as API tokens, use `--secret` instead, so the value is -resolved from your secrets provider rather than passed in plaintext. - -### Run a server within a group - -To run an MCP server within a specific group, use the `--group` option. This -allows you to organize your servers and manage them collectively. - -```bash -thv run --group -``` - -:::note - -The group must exist before you can run a server in it. - -::: - -See [Organize servers into groups](./group-management.mdx) to learn more about -organizing servers and configuring client access. - -### Mount a local file or directory - -To enable file system access for an MCP server, you can either use the -`--volume` flag to mount specific paths or create a custom permission profile -that defines read and write permissions. - -See [File system access](./filesystem-access.mdx) for detailed examples. To -prevent sensitive files from being exposed when mounting a project, use -[.thvignore](./thvignore.mdx). - -### Restrict network access - -ToolHive enables _network isolation_ by default, restricting an MCP server's -outbound traffic to the hosts and ports declared in its permission profile. -Override the default profile with `--permission-profile`, or opt out entirely -with `--isolate-network=false`. - -See [Network isolation](./network-isolation.mdx) for network architecture -details and examples. - -### Add command-line arguments - -Some MCP servers require additional arguments to run correctly. You can pass -these arguments after the server name in the -[`thv run`](../reference/cli/thv_run.md) command: - -```bash -thv run -- -``` - -For example: - -```bash -thv run my-mcp-server:latest -- --arg1 value1 --arg2 value2 -``` - -Check the MCP server's documentation for the required arguments. - -:::warning - -Some MCP servers in the ToolHive registry include default arguments that are -essential for proper operation. When you provide custom arguments using -`-- `, these replace the registry defaults entirely rather than adding to -them. - -Before adding custom arguments, check the server's registry entry: - -```bash -thv registry info --format json | jq '.args' -``` - -If default arguments are listed, include them along with your custom arguments -to ensure the server functions correctly. - -::: - -### Enable audit logging - -ToolHive can record an audit trail of the MCP activity that flows through a -server's proxy. Audit events capture interactions such as client initialization, -`tools/list` and `tools/call` requests, resource reads, prompt requests, and -notifications, along with metadata like the timestamp, component name, and -outcome. - -To turn on audit logging with ToolHive's built-in defaults, use the -`--enable-audit` flag: - -```bash -thv run --enable-audit -``` - -The default configuration records each event type but omits the request and -response payloads to protect potentially sensitive data. Audit events are -written to the server's logs, so you can view them with `thv logs`: - -```bash -thv logs -``` - -Each event is emitted as a JSON object on its own line, which makes the output -easy to filter or forward to a log aggregator. - -For finer control, supply your own configuration file with the `--audit-config` -flag instead: - -```bash -thv run --audit-config ./audit.json -``` - -A configuration file lets you select which event types to record with -`eventTypes` (or exclude specific types with `excludeEventTypes`), choose -whether to include request or response data, and set where to send the audit -log. For example: - -```json title="audit.json" -{ - "component": "fetch-server", - "eventTypes": ["mcp_tool_call", "mcp_resource_read"], - "includeRequestData": true, - "includeResponseData": false, - "maxDataSize": 2048, - "logFile": "./fetch-audit.log" -} -``` - -When you set `logFile`, ToolHive appends audit events to that file instead of -the server's standard logs. Use a path that the user running `thv` can write to. -Leave it unset to keep events in the logs viewed by `thv logs`. - -:::note - -Use either flag to turn on audit logging: `--enable-audit` for the built-in -defaults, or `--audit-config` to load your own settings. `--audit-config` -enables auditing on its own, so you don't need to pass `--enable-audit` as well. - -::: - -Setting `includeRequestData` or `includeResponseData` to `true` can capture -sensitive arguments or results in the audit log. Enable these options only when -you understand the privacy implications, and adjust `maxDataSize` to limit how -much of each payload is recorded. - -### Verify container image signatures - -For MCP servers that run from a container image, ToolHive can check the image's -signature and provenance before launching it. Some registry servers publish -provenance information that ToolHive uses to confirm the image was built and -signed as expected. To see whether a given server has it, check the -`Has Provenance` field in the output of `thv registry info `. - -Use the `--image-verification` flag to control this behavior. It accepts three -values: - -- `warn` (default): verify the image and log a warning if verification fails or - the server has no provenance information, but run the server anyway. -- `enabled`: verify the image and refuse to run the server if verification fails - or no provenance information is available. -- `disabled`: skip image verification entirely. - -```bash -thv run --image-verification enabled -``` - -For example, to enforce verification when running the `fetch` server: - -```bash -thv run --image-verification enabled fetch -``` - -:::note - -Verification succeeds only for servers that include provenance information, and -not every server does. When a server has no provenance data, `warn` runs it with -a warning while `enabled` declines to run it. If you set `enabled`, keep in mind -that it stops servers without provenance, not only servers that fail -verification. - -::: - -### Run a server on a specific port - -ToolHive creates a reverse proxy on a random port that forwards requests to the -container. This is the port that client applications connect to. To set a -specific proxy port instead, use the `--proxy-port` flag: - -```bash -thv run --proxy-port -``` - -To publish a container's own port directly to the host (separate from the proxy -port), use `--publish` (`-p`) with `hostPort:containerPort`. Repeat the flag for -multiple ports: - -```bash -thv run --publish 8090:8080 -``` - -Most servers only need the proxy port; use `--publish` when a server exposes an -additional port you need to reach directly. - -### Override the session timeout - -ToolHive's proxy evicts idle MCP sessions after 2 hours by default. To raise or -lower this inactivity timeout for a workload, pass `--session-ttl` with a Go -duration string: - -```bash -thv run --session-ttl 4h -``` - -Set a longer value when clients hold sessions open for long-running operations, -or a shorter value to free resources faster. - -### Run a server in the foreground - -By default, ToolHive runs the server in the background and returns control to -your shell. To keep the process attached to your terminal until the container -exits, use the `--foreground` (`-f`) flag: - -```bash -thv run --foreground -``` - -This is useful for debugging or when you want the server's lifecycle tied to the -current shell session. - -### Run a server exposing only selected tools - -ToolHive can filter the tools returned to the client as result of a `tools/list` -command as well as block calls to tools that you don't want to expose. - -This can help reduce the amount of tools sent to the LLM while still using the -same MCP server, but it is not meant as a security feature. - -To filter the list of tools, use the `--tools` flag either once - -```bash -thv run --tools -``` - -Or multiple times - -```bash -thv run --tools --tools -``` +### Run servers in a group -For example: +You can organize MCP servers into groups using `thv group create` and then run +servers within a group: ```bash -thv run --tools list_issues --tools get_issue github -``` - -If the server comes from the registry, ToolHive can validate the tool names -against the list advertised in the image reference. An error is returned in case -ToolHive cannot find one of the specified tools. - -### Override tool names and descriptions - -With ToolHive you can modify how tools exposed by an MCP server are exposed to -clients. In particular, tool names and descriptions can be changed. - -This is useful when you want to guide an agent toward calling a specific tool -for particular questions. - -One common use case is running multiple copies of the same MCP server with -different [network access levels](./network-isolation.mdx), one for internal -resources and another for public internet access. By overriding the tool names -and descriptions, you can help your agent choose the right server for each task. - -For example, the `fetch` MCP server exposes a single `fetch` tool with a -description like: - -``` -"Fetches a URL from the internet and optionally extracts its contents as markdown." -``` - -To override this, create a configuration file with one entry under -`toolsOverride` for each tool you want to modify: - -```json -{ - "toolsOverride": { - "fetch": { - "name": "toolhive-docs-fetch", - "description": "Fetches a URL from https://docs.stacklok.com/toolhive website." - } - } -} -``` - -Then pass this file to [`thv run`](../reference/cli/thv_run.md): - -```sh -thv run --tools-override override.json fetch +thv group create my-group +thv run --group my-group fetch +thv run --group my-group github ``` -The key in the override object is the _original tool name_, while the `name` -field contains the _new name_ that clients will see. - -:::info - -Take care when using `--tools` and `--tools-override` together in the same -command. - -Tool filtering and tool overrides work independently: filtering limits access to -a subset of tools, while overrides change how those tools appear to clients. +See [Group management](./group-management.mdx) for more details on creating and +managing groups. -When using both options, `--tools` must reference the _overridden names_ (the -new names you define) since those are what clients will see. +See [Run a custom MCP server](#run-a-custom-mcp-server) to run a server that is +not in the registry, or [Run remote MCP servers](./run-remote-mcp-servers.mdx) +for more details about remote server configuration. -::: +To customize a server's behavior, such as setting a custom name, passing +secrets, or filtering tools, see +[Configure MCP servers](./configure-mcp-servers.mdx). ## Run a custom MCP server @@ -791,291 +411,6 @@ thv run uvx://some-package thv run --ca-cert /path/to/special-ca.crt uvx://other-package ``` -## Run a remote MCP server - -You can run remote MCP servers directly by providing their URL. This allows you -to connect to MCP servers hosted elsewhere without needing to manage containers -locally. ToolHive creates a transparent proxy that handles authentication and -forwards requests to the remote server. - -### Basic remote server setup - -To run a remote MCP server, simply provide its URL: - -```bash -thv run [--name ] -``` - -For example: - -```bash -thv run https://api.example.com/mcp -``` - -If you don't specify a name with `--name`, ToolHive will automatically derive a -name from the URL by extracting the main domain name (e.g., `notion` from -`https://api.notion.com/mcp`). - -By default, remote servers use the `streamable-http` transport. If the server -uses Server-Sent Events (SSE), specify the transport flag: - -```bash -thv run https://api.example.com/sse --transport sse -``` - -#### Stateless remote servers - -Some remote MCP servers only accept POST requests and don't support SSE streams. -For these servers, use the `--stateless` flag: - -```bash -thv run https://example.com/mcp --name my-server --transport streamable-http --stateless -``` - -When `--stateless` is set: - -- GET, HEAD, and DELETE requests return HTTP 405 at the proxy instead of being - forwarded to the remote server -- Health checks use a POST-based JSON-RPC ping instead of SSE - -:::info[What's happening?] - -When you run a remote MCP server, ToolHive: - -1. Automatically detects if the remote server requires authentication. -2. Handles OAuth/OIDC authentication flows if needed. -3. Starts an HTTP proxy process on a random port to forward client requests to - the remote server. -4. Manages the server like any other ToolHive workload. No container is created - for remote MCP servers. - -::: - -### Authentication setup - -Many remote MCP servers require authentication. ToolHive supports automatic -authentication detection and OAuth/OIDC flows. - -#### Auto-detect authentication - -ToolHive can automatically detect if a remote server requires authentication by -examining the server's response headers and status codes: - -```bash -thv run https://protected-api.com/mcp --name my-server -``` - -If authentication is required, ToolHive will prompt you to complete the OAuth -flow. When no client credentials are provided, ToolHive identifies itself to the -authorization server using one of two mechanisms: - -- **Client ID Metadata Document (CIMD).** If the authorization server's - discovery document sets `client_id_metadata_document_supported: true`, - ToolHive presents - [`https://toolhive.dev/oauth/client-metadata.json`](https://toolhive.dev/oauth/client-metadata.json) - as its `client_id`. No registration round-trip is needed. -- **Dynamic Client Registration (DCR, RFC 7591).** When CIMD is unavailable or - the authorization server rejects the CIMD `client_id` with - `invalid_client`/`unauthorized_client`, ToolHive falls back to registering an - OAuth client dynamically. - -Either path eliminates the need to pre-configure a client ID and secret for -authorization servers that support them. - -#### Bearer token authentication - -Some remote MCP servers accept a static bearer token in the `Authorization` -header instead of a full OAuth flow. Use `--remote-auth-bearer-token` to provide -the token directly: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-auth-bearer-token -``` - -ToolHive sends the value as an `Authorization: Bearer ` header on every -request forwarded to the remote server. The token is stored in ToolHive's -secrets manager; only a reference to it is saved in the run configuration, never -the token itself. - -To keep the token out of your shell history and process list, store it in a file -and reference it with `--remote-auth-bearer-token-file`: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-auth-bearer-token-file ./token.txt -``` - -For servers that expect the credential in a different header, such as -`X-API-Key`, use [forwarded headers](#inject-custom-headers) instead. - -#### OIDC authentication - -For servers using OpenID Connect (OIDC), provide the issuer URL, client ID, and -client secret obtained from the application provider: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-auth-issuer https://auth.example.com \ - --remote-auth-client-id my-client-id \ - --remote-auth-client-secret my-client-secret -``` - -#### OAuth2 authentication - -For servers using OAuth2, specify the authorization and token URLs along with -the client ID and secret obtained from the application provider: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-auth-authorize-url https://auth.example.com/oauth/authorize \ - --remote-auth-token-url https://auth.example.com/oauth/token \ - --remote-auth-client-id my-client-id \ - --remote-auth-client-secret my-client-secret -``` - -#### Automatic token refresh - -ToolHive automatically handles OAuth token refresh for remote MCP servers. When -you authenticate with a remote server, ToolHive stores both the access token and -refresh token securely. The refresh token is used to automatically obtain new -access tokens when they expire, ensuring uninterrupted service without requiring -manual re-authentication. - -### Advanced remote server configuration - -#### OAuth scopes and parameters - -Specify custom OAuth scopes for the authentication flow: - -```bash -thv run https://api.example.com/mcp \ - ... \ - --remote-auth-scopes read,write,admin -``` - -#### Non-standard OAuth scope parameter name - -Some OAuth providers use a non-standard query parameter name for scopes in the -authorization URL. For example, Slack uses `user_scope` instead of the standard -`scope` parameter. Use the `--remote-auth-scope-param-name` flag to override the -parameter name: - -```bash -thv run https://mcp.slack.com/mcp \ - --name slack \ - --transport streamable-http \ - --remote-auth-client-id \ - --remote-auth-callback-port 3118 \ - --remote-auth-authorize-url "https://slack.com/oauth/v2_user/authorize" \ - --remote-auth-token-url "https://slack.com/api/oauth.v2.user.access" \ - --remote-auth-scope-param-name user_scope -``` - -#### Resource indicator (RFC 8707) - -When authenticating to remote MCP servers, you can specify a resource indicator -as defined by [RFC 8707](https://datatracker.ietf.org/doc/html/rfc8707). This -allows the authorization server to return an access token with a scoped -audience, which will then be passed to and validated by the remote MCP server. - -ToolHive does not auto-derive the resource indicator from the server URL by -default. Set it explicitly with `--remote-auth-resource` when the authorization -server requires it: - -```bash -thv run https://api.example.com/mcp \ - ... \ - --remote-auth-resource https://api.example.com -``` - -The resource parameter must include a scheme and host, and cannot contain -fragments. If you provide an invalid resource parameter, ToolHive will return an -error. - -#### Custom authentication timeout - -Adjust the authentication timeout for slow networks: - -```bash -thv run https://api.example.com/mcp \ - ... \ - --remote-auth-timeout 2m -``` - -#### Skip browser authentication - -For headless environments, skip the browser-based OAuth flow: - -```bash -thv run https://api.example.com/mcp \ - ... \ - --remote-auth-skip-browser -``` - -#### Inject custom headers - -Some remote MCP servers require custom headers for tenant identification, API -keys, or other purposes. ToolHive can inject headers into every request -forwarded to the remote server. - -To add plaintext headers, use the `--remote-forward-headers` flag: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-forward-headers "X-Tenant-ID=tenant123" \ - --remote-forward-headers "X-Custom-Header=value" -``` - -For sensitive values like API keys, use the `--remote-forward-headers-secret` -flag to reference values stored in ToolHive's secrets manager: - -```bash -# First, store the secret (enter the value when prompted) -thv secret set my-api-key - -# Then reference it by name -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-forward-headers-secret "X-Api-Key=my-api-key" -``` - -You can combine plaintext and secret-backed headers in a single command: - -```bash -thv run https://api.example.com/mcp \ - --name my-server \ - --remote-forward-headers "X-Tenant-ID=tenant123" \ - --remote-forward-headers-secret "X-Api-Key=my-api-key" -``` - -:::warning[Security considerations] - -- Plaintext header values are stored in the server's configuration file. For - sensitive values (API keys, tokens), always use - `--remote-forward-headers-secret`. -- Secret-backed header values are resolved at runtime and never stored in - configuration files. -- Certain headers cannot be configured for security reasons, including `Host`, - `Connection`, `Transfer-Encoding`, and proxy-related headers like - `X-Forwarded-For`. - -::: - -### Remote server management - -Remote MCP servers are managed like any other ToolHive workload: - -- **List servers**: `thv list` shows remote servers with their target URLs -- **View logs**: `thv logs ` shows proxy and authentication logs -- **Stop/restart**: `thv stop ` and `thv restart ` -- **Remove**: `thv rm ` removes the proxy and configuration - ## Share and reuse server configurations ToolHive allows you to export a server's configuration and run servers using @@ -1193,13 +528,12 @@ The configuration file must contain all server settings. - [Test your MCP servers](./test-mcp-servers.mdx) using the MCP Inspector or `thv mcp` commands - [Secure your servers](./auth.mdx) with OIDC authentication and Cedar policies -- [Enable telemetry and metrics](./telemetry-and-metrics.mdx) to gain - observability into MCP server interactions -- [Run the API server](./api-server.mdx) for programmatic access to ToolHive ## Related information - [`thv run` command reference](../reference/cli/thv_run.md) +- [Configure MCP servers](./configure-mcp-servers.mdx) +- [Run remote MCP servers](./run-remote-mcp-servers.mdx) - [Client configuration](./client-configuration.mdx) - [Secrets management](./secrets-management.mdx) - [Custom permissions](./custom-permissions.mdx) @@ -1306,62 +640,6 @@ If a server crashes or exits unexpectedly: -
-Remote server authentication fails - -If a remote MCP server authentication fails: - -1. Check the server logs for authentication errors (see - [View server logs](./manage-mcp-servers.mdx#view-server-logs) for the correct - log file path on your platform). - -2. Verify the issuer URL is correct and reachable from your machine. - -3. Confirm the OAuth client ID and secret are valid and that the redirect URI - matches what's registered with your identity provider. - -4. Re-authenticate by restarting the server: - - ```bash - thv restart - ``` - -
- -
-Remote server connection issues - -If you can't connect to a remote MCP server: - -1. Confirm the remote server URL is correct and reachable from your machine: - - ```bash - curl -I https://api.example.com/mcp - ``` - -2. Check what the remote server is actually advertising through ToolHive. For - remote-backed servers, pass the proxy URL from `thv list` rather than the - server name (the `--server` flag currently resolves names only for local - servers; see [Test MCP servers](./test-mcp-servers.mdx)): - - ```bash - thv mcp list tools --server - ``` - -3. Review the server logs for connection or authentication errors: - - ```bash - thv logs - ``` - -4. Re-run with debug logging if you need more detail: - - ```bash - thv run --debug https://api.example.com/mcp --name my-server - ``` - -
-
MCP server can't connect to services on the same host diff --git a/docs/toolhive/guides-cli/run-remote-mcp-servers.mdx b/docs/toolhive/guides-cli/run-remote-mcp-servers.mdx new file mode 100644 index 00000000..22533ccd --- /dev/null +++ b/docs/toolhive/guides-cli/run-remote-mcp-servers.mdx @@ -0,0 +1,378 @@ +--- +title: Run remote MCP servers +description: + Connect to remotely hosted MCP servers with the ToolHive CLI, including OAuth, + OIDC, and bearer token authentication. +--- + +You can run remote MCP servers directly by providing their URL. This allows you +to connect to MCP servers hosted elsewhere without needing to manage containers +locally. ToolHive creates a transparent proxy that handles authentication and +forwards requests to the remote server. + +:::note[Remote servers from the registry] + +If a remote server is published in the [ToolHive registry](./registry.mdx), you +can run it by name with the pre-configured URL and authentication settings. See +[Run a server from the registry](./run-mcp-servers.mdx#run-a-server-from-the-registry). +This guide covers running remote servers by URL. + +::: + +## Basic remote server setup + +To run a remote MCP server, simply provide its URL: + +```bash +thv run [--name ] +``` + +For example: + +```bash +thv run https://api.example.com/mcp +``` + +If you don't specify a name with `--name`, ToolHive will automatically derive a +name from the URL by extracting the main domain name (e.g., `notion` from +`https://api.notion.com/mcp`). + +By default, remote servers use the `streamable-http` transport. If the server +uses Server-Sent Events (SSE), specify the transport flag: + +```bash +thv run https://api.example.com/sse --transport sse +``` + +:::info[What's happening?] + +When you run a remote MCP server, ToolHive: + +1. Automatically detects if the remote server requires authentication. +2. Handles OAuth/OIDC authentication flows if needed. +3. Starts an HTTP proxy process on a random port to forward client requests to + the remote server. +4. Manages the server like any other ToolHive workload. No container is created + for remote MCP servers. + +::: + +### Stateless remote servers + +Some remote MCP servers only accept POST requests and don't support SSE streams. +For these servers, use the `--stateless` flag: + +```bash +thv run https://example.com/mcp --name my-server --transport streamable-http --stateless +``` + +When `--stateless` is set: + +- GET, HEAD, and DELETE requests return HTTP 405 at the proxy instead of being + forwarded to the remote server +- Health checks use a POST-based JSON-RPC ping instead of SSE + +## Authentication setup + +Most remote MCP servers require authentication. ToolHive supports automatic +authentication detection and OAuth/OIDC flows. + +### Auto-detect authentication + +ToolHive can automatically detect if a remote server requires authentication by +examining the server's response headers and status codes: + +```bash +thv run https://protected-api.com/mcp --name my-server +``` + +If authentication is required, ToolHive will prompt you to complete the OAuth +flow. When no client credentials are provided, ToolHive identifies itself to the +authorization server using one of two mechanisms: + +- **Client ID Metadata Document (CIMD).** If the authorization server's + discovery document sets `client_id_metadata_document_supported: true`, + ToolHive presents + [`https://toolhive.dev/oauth/client-metadata.json`](https://toolhive.dev/oauth/client-metadata.json) + as its `client_id`. No registration round-trip is needed. +- **Dynamic Client Registration (DCR, RFC 7591).** When CIMD is unavailable or + the authorization server rejects the CIMD `client_id` with + `invalid_client`/`unauthorized_client`, ToolHive falls back to registering an + OAuth client dynamically. + +Either path eliminates the need to pre-configure a client ID and secret for +authorization servers that support them. + +### Bearer token authentication + +Some remote MCP servers accept a static bearer token in the `Authorization` +header instead of a full OAuth flow. Use `--remote-auth-bearer-token` to provide +the token directly: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-auth-bearer-token +``` + +ToolHive sends the value as an `Authorization: Bearer ` header on every +request forwarded to the remote server. The token is stored in ToolHive's +secrets manager; only a reference to it is saved in the run configuration, never +the token itself. + +To keep the token out of your shell history and process list, store it in a file +and reference it with `--remote-auth-bearer-token-file`: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-auth-bearer-token-file ./token.txt +``` + +For servers that expect the credential in a different header, such as +`X-API-Key`, use [forwarded headers](#inject-custom-headers) instead. + +### OIDC authentication + +For servers using OpenID Connect (OIDC), provide the issuer URL, client ID, and +client secret obtained from the application provider: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-auth-issuer https://auth.example.com \ + --remote-auth-client-id my-client-id \ + --remote-auth-client-secret my-client-secret +``` + +To keep the client secret out of your shell history and process list, store it +in a file and reference it with `--remote-auth-client-secret-file` instead. + +### OAuth2 authentication + +For servers using OAuth2, specify the authorization and token URLs along with +the client ID and secret obtained from the application provider: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-auth-authorize-url https://auth.example.com/oauth/authorize \ + --remote-auth-token-url https://auth.example.com/oauth/token \ + --remote-auth-client-id my-client-id \ + --remote-auth-client-secret my-client-secret +``` + +As with OIDC, use `--remote-auth-client-secret-file` to read the client secret +from a file instead of passing it on the command line. + +### Automatic token refresh + +ToolHive automatically handles OAuth token refresh for remote MCP servers. When +you authenticate with a remote server, ToolHive stores both the access token and +refresh token securely. The refresh token is used to automatically obtain new +access tokens when they expire, ensuring uninterrupted service without requiring +manual re-authentication. + +## Advanced remote server configuration + +### OAuth scopes and parameters + +Specify custom OAuth scopes for the authentication flow: + +```bash +thv run https://api.example.com/mcp \ + ... \ + --remote-auth-scopes read,write,admin +``` + +### Non-standard OAuth scope parameter name + +Some OAuth providers use a non-standard query parameter name for scopes in the +authorization URL. For example, Slack uses `user_scope` instead of the standard +`scope` parameter. Use the `--remote-auth-scope-param-name` flag to override the +parameter name: + +```bash +thv run https://mcp.slack.com/mcp \ + --name slack \ + --transport streamable-http \ + --remote-auth-client-id \ + --remote-auth-callback-port 3118 \ + --remote-auth-authorize-url "https://slack.com/oauth/v2_user/authorize" \ + --remote-auth-token-url "https://slack.com/api/oauth.v2.user.access" \ + --remote-auth-scope-param-name user_scope +``` + +### Resource indicator (RFC 8707) + +When authenticating to remote MCP servers, you can specify a resource indicator +as defined by [RFC 8707](https://datatracker.ietf.org/doc/html/rfc8707). This +allows the authorization server to return an access token with a scoped +audience, which will then be passed to and validated by the remote MCP server. + +ToolHive does not auto-derive the resource indicator from the server URL by +default. Set it explicitly with `--remote-auth-resource` when the authorization +server requires it: + +```bash +thv run https://api.example.com/mcp \ + ... \ + --remote-auth-resource https://api.example.com +``` + +The resource parameter must include a scheme and host, and cannot contain +fragments. If you provide an invalid resource parameter, ToolHive will return an +error. + +### Custom authentication timeout + +Adjust the authentication timeout for slow networks: + +```bash +thv run https://api.example.com/mcp \ + ... \ + --remote-auth-timeout 2m +``` + +### Skip browser authentication + +For headless environments, skip the browser-based OAuth flow: + +```bash +thv run https://api.example.com/mcp \ + ... \ + --remote-auth-skip-browser +``` + +### Inject custom headers + +Some remote MCP servers require custom headers for tenant identification, API +keys, or other purposes. ToolHive can inject headers into every request +forwarded to the remote server. + +To add plaintext headers, use the `--remote-forward-headers` flag: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-forward-headers "X-Tenant-ID=tenant123" \ + --remote-forward-headers "X-Custom-Header=value" +``` + +For sensitive values like API keys, use the `--remote-forward-headers-secret` +flag to reference values stored in ToolHive's secrets manager: + +```bash +# First, store the secret (enter the value when prompted) +thv secret set my-api-key + +# Then reference it by name +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-forward-headers-secret "X-Api-Key=my-api-key" +``` + +You can combine plaintext and secret-backed headers in a single command: + +```bash +thv run https://api.example.com/mcp \ + --name my-server \ + --remote-forward-headers "X-Tenant-ID=tenant123" \ + --remote-forward-headers-secret "X-Api-Key=my-api-key" +``` + +:::warning[Security considerations] + +- Plaintext header values are stored in the server's configuration file. For + sensitive values (API keys, tokens), always use + `--remote-forward-headers-secret`. +- Secret-backed header values are resolved at runtime and never stored in + configuration files. +- Certain headers cannot be configured for security reasons, including `Host`, + `Connection`, `Transfer-Encoding`, and proxy-related headers like + `X-Forwarded-For`. + +::: + +## Remote server management + +Remote MCP servers are managed like any other ToolHive workload: + +- **List servers**: `thv list` shows remote servers with their target URLs +- **View logs**: `thv logs ` shows proxy and authentication logs +- **Stop/restart**: `thv stop ` and `thv restart ` +- **Remove**: `thv rm ` removes the proxy and configuration + +## Next steps + +- [Configure MCP servers](./configure-mcp-servers.mdx) to customize names, + ports, tool filtering, and other shared options +- [Monitor and manage MCP servers](./manage-mcp-servers.mdx) to control your + running servers +- [Secure your servers](./auth.mdx) with OIDC authentication and Cedar policies + +## Related information + +- [Run MCP servers](./run-mcp-servers.mdx) +- [`thv run` command reference](../reference/cli/thv_run.md) +- [Enable telemetry and metrics](./telemetry-and-metrics.mdx) +- [Secrets management](./secrets-management.mdx) +- [Run remote MCP servers in Kubernetes](../guides-k8s/remote-mcp-proxy.mdx) + +## Troubleshooting + +
+Remote server authentication fails + +If a remote MCP server authentication fails: + +1. Check the server logs for authentication errors (see + [View server logs](./manage-mcp-servers.mdx#view-server-logs) for the correct + log file path on your platform). + +2. Verify the issuer URL is correct and reachable from your machine. + +3. Confirm the OAuth client ID and secret are valid and that the redirect URI + matches what's registered with your identity provider. + +4. Re-authenticate by restarting the server: + + ```bash + thv restart + ``` + +
+ +
+Remote server connection issues + +If you can't connect to a remote MCP server: + +1. Confirm the remote server URL is correct and reachable from your machine: + + ```bash + curl -I https://api.example.com/mcp + ``` + +2. Check what the remote server is actually advertising through ToolHive. For + remote-backed servers, pass the proxy URL from `thv list` rather than the + server name (the `--server` flag currently resolves names only for local + servers; see [Test MCP servers](./test-mcp-servers.mdx)): + + ```bash + thv mcp list tools --server + ``` + +3. Review the server logs for connection or authentication errors: + + ```bash + thv logs + ``` + +4. Re-run with debug logging if you need more detail: + + ```bash + thv run --debug https://api.example.com/mcp --name my-server + ``` + +
diff --git a/docs/toolhive/guides-k8s/connect-clients.mdx b/docs/toolhive/guides-k8s/connect-clients.mdx index 3166ac40..f5473eb6 100644 --- a/docs/toolhive/guides-k8s/connect-clients.mdx +++ b/docs/toolhive/guides-k8s/connect-clients.mdx @@ -736,8 +736,8 @@ thv run --name fetch-k8s https://mcp.example.com/fetch/mcp ``` If authentication is configured, add the appropriate flags. See the -[ToolHive CLI guide](../guides-cli/run-mcp-servers.mdx#authentication-setup) for -details. +[ToolHive CLI guide](../guides-cli/run-remote-mcp-servers.mdx#authentication-setup) +for details. The MCP server is now available to your configured MCP clients. @@ -747,7 +747,7 @@ The MCP server is now available to your configured MCP clients. For more details on using remote MCP servers, see: - [Run remote MCP servers (UI)](../guides-ui/run-mcp-servers.mdx#custom-remote-servers) -- [Run remote MCP servers (CLI)](../guides-cli/run-mcp-servers.mdx#run-a-remote-mcp-server) +- [Run remote MCP servers (CLI)](../guides-cli/run-remote-mcp-servers.mdx) ## Connect from within the cluster diff --git a/sidebars.ts b/sidebars.ts index 992cb179..33d94b39 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -97,6 +97,8 @@ const mcpSidebar: SidebarsConfig[string] = [ id: 'toolhive/guides-cli/run-mcp-servers', }, items: [ + 'toolhive/guides-cli/run-remote-mcp-servers', + 'toolhive/guides-cli/configure-mcp-servers', 'toolhive/guides-cli/manage-mcp-servers', 'toolhive/guides-cli/group-management', 'toolhive/guides-cli/secrets-management',