Updates from mdx files

docs-site/content/kagent/supported-providers/azure-ai-foundry.md

@@ -0,0 +1,257 @@
+---
+title: Azure AI Foundry
+description: Learn how to configure Azure AI Foundry models in kagent.
+weight: 3
+author: kagent.dev
+---
+
+## Configuring Azure AI Foundry
+
+> **Note:** Foundry chat models and memory embeddings require the Go agent runtime (`runtime: go`). Chat models must be available through Foundry's OpenAI-compatible chat completions API. Claude and other models are not yet supported.
+
+The following steps use API key authentication. To authenticate without an API key, see [Workload Identity](#workload-identity).
+
+1. Create a Kubernetes Secret that contains your Foundry API key.
+
+```bash
+export FOUNDRY_API_KEY="<your-foundry-api-key>"
+
+kubectl create secret generic foundry-api-key \
+  --namespace kagent \
+  --from-literal=api-key="${FOUNDRY_API_KEY}"
+```
+
+2. Create a `ModelConfig` for your Foundry deployment.
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: foundry-chat
+  namespace: kagent
+spec:
+  provider: Foundry
+  model: gpt-4.1-nano
+  apiKeySecret: foundry-api-key
+  apiKeySecretKey: api-key
+  foundry:
+    endpoint: https://my-foundry-account.cognitiveservices.azure.com/
+    deployment: gpt-4-1-nano
+    apiVersion: "2024-10-21"
+```
+
+3. Reference the `ModelConfig` from an agent that uses the Go runtime.
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: Agent
+metadata:
+  name: foundry-agent
+  namespace: kagent
+spec:
+  type: Declarative
+  declarative:
+    runtime: go
+    modelConfig: foundry-chat
+    systemMessage: "You are a helpful assistant."
+```
+
+4. Save the manifests from steps 2 and 3 as `foundry-model.yaml` and `foundry-agent.yaml`, then apply them to your cluster.
+
+```bash
+kubectl apply -f foundry-model.yaml
+kubectl apply -f foundry-agent.yaml
+```
+
+Alternatively, apply only `foundry-model.yaml`, then select the `ModelConfig` from the **Model** dropdown when you create or update an agent in the kagent UI.
+
+## ModelConfig reference
+
+| Field | Required | Description |
+| --- | --- | --- |
+| `spec.provider` | Always | Must be `Foundry`. |
+| `spec.model` | Always | Model name reported to the runtime, such as `gpt-4.1-nano`. This can differ from the Azure deployment name. |
+| `spec.foundry.endpoint` | Exactly one endpoint field | Account endpoint, such as `https://<account>.cognitiveservices.azure.com/`. |
+| `spec.foundry.endpointFrom` | Exactly one endpoint field | Resolve the endpoint from a ConfigMap key. See [Endpoint from a ConfigMap](#endpoint-from-a-configmap). |
+| `spec.foundry.deployment` | Always | Foundry model deployment name. |
+| `spec.foundry.apiVersion` | Optional | Azure AI Foundry data-plane API version. Defaults to `2024-10-21`. |
+| `spec.apiKeySecret` | Optional | Secret that contains the API key. Omit both this field and `apiKeyPassthrough` to use Workload Identity. Mutually exclusive with `apiKeyPassthrough`. |
+| `spec.apiKeySecretKey` | With `apiKeySecret` | Key within `apiKeySecret` that contains the API key. |
+| `spec.apiKeyPassthrough` | Optional | Let each caller supply its own Foundry API key instead of using a shared Secret. Mutually exclusive with `apiKeySecret`. See [Token passthrough](#token-passthrough). |
+
+## Authentication
+
+The runtime chooses a credential based on the fields in the `ModelConfig`.
+
+| Configuration | Credential |
+| --- | --- |
+| `apiKeySecret` is set | API key from the referenced Secret. |
+| `apiKeyPassthrough: true` | Foundry API key supplied by the caller's A2A request. |
+| Neither field is set | Azure Workload Identity. |
+
+### Workload Identity
+
+Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime tries to authenticate using your Azure CLI login.
+
+> **Note:** The runtime validates Azure credentials at startup. If credentials cannot be resolved, the agent does not become ready.
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: foundry-chat
+  namespace: kagent
+spec:
+  provider: Foundry
+  model: gpt-4.1-nano
+  foundry:
+    endpoint: https://my-foundry-account.cognitiveservices.azure.com/
+    deployment: gpt-4-1-nano
+    apiVersion: "2024-10-21"
+```
+
+See [Configure the agent for Azure Workload Identity](#configure-the-agent-for-azure-workload-identity) for the required agent settings.
+
+### Token passthrough
+
+Set `apiKeyPassthrough: true`, then send the Foundry API key as the bearer token in each Agent2Agent (A2A) request. The runtime forwards that value to Foundry as the API key. Use [Workload Identity](#workload-identity), not token passthrough, for Microsoft Entra ID authentication.
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: foundry-passthrough
+  namespace: kagent
+spec:
+  provider: Foundry
+  model: gpt-4.1-nano
+  apiKeyPassthrough: true
+  foundry:
+    endpoint: https://my-foundry-account.cognitiveservices.azure.com/
+    deployment: gpt-4-1-nano
+```
+
+## Endpoint from a ConfigMap
+
+To use an endpoint from a ConfigMap, set `foundry.endpointFrom` to the ConfigMap name and key. For example, [Azure Service Operator](https://azure.github.io/azure-service-operator/) (ASO) can provision the account and write its endpoint to a ConfigMap.
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: foundry-account
+  namespace: kagent
+data:
+  endpoint: https://my-foundry-account.cognitiveservices.azure.com/
+---
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: foundry-chat
+  namespace: kagent
+spec:
+  provider: Foundry
+  model: gpt-4.1-nano
+  foundry:
+    endpointFrom:
+      name: foundry-account
+      key: endpoint
+    deployment: gpt-4-1-nano
+    apiVersion: "2024-10-21"
+```
+
+The ConfigMap must be in the same namespace as the `ModelConfig`.
+
+> **Note:** Updating the endpoint value triggers a rolling update of agent pods that use this `ModelConfig` as their primary model.
+
+## Configure the agent for Azure Workload Identity
+
+First, follow the [AKS Workload Identity deployment guide](https://learn.microsoft.com/azure/aks/workload-identity-deploy-cluster) to configure your cluster and managed identity. Grant the identity the `Cognitive Services User` role on the Foundry resource.
+
+Azure Workload Identity requires the `azure.workload.identity/use: "true"` label on the agent pod and the managed identity client ID on its ServiceAccount. Configure the agent using one of the following options.
+
+### Let kagent create the ServiceAccount
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: Agent
+metadata:
+  name: foundry-agent
+  namespace: kagent
+spec:
+  type: Declarative
+  declarative:
+    runtime: go
+    modelConfig: foundry-chat
+    systemMessage: "You are a helpful assistant."
+    deployment:
+      labels:
+        azure.workload.identity/use: "true"
+      serviceAccountConfig:
+        annotations:
+          azure.workload.identity/client-id: <managed-identity-client-id>
+```
+
+The ServiceAccount has the same name and namespace as the agent. Therefore, use `system:serviceaccount:<namespace>:<agent-name>` as the subject of the Azure federated identity credential.
+
+> **Note:** If you use Helm to configure a shared ServiceAccount with `controller.agentDeployment.serviceAccountName`, annotate that ServiceAccount and follow [Use an existing ServiceAccount](#use-an-existing-serviceaccount).
+
+### Use an existing ServiceAccount
+
+Create or reuse a pre-annotated ServiceAccount, then reference it from the agent. The pod label is still required.
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: foundry-workload-identity
+  namespace: kagent
+  annotations:
+    azure.workload.identity/client-id: <managed-identity-client-id>
+---
+apiVersion: kagent.dev/v1alpha2
+kind: Agent
+metadata:
+  name: foundry-agent
+  namespace: kagent
+spec:
+  type: Declarative
+  declarative:
+    runtime: go
+    modelConfig: foundry-chat
+    systemMessage: "You are a helpful assistant."
+    deployment:
+      serviceAccountName: foundry-workload-identity
+      labels:
+        azure.workload.identity/use: "true"
+```
+
+Use `system:serviceaccount:<namespace>:<service-account-name>` as the subject of the Azure federated identity credential.
+
+## Memory embeddings
+
+Configure a memory embedding `ModelConfig` for a Foundry embedding deployment.
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: foundry-embeddings
+  namespace: kagent
+spec:
+  provider: Foundry
+  model: text-embedding-3-small
+  foundry:
+    endpoint: https://my-foundry-account.cognitiveservices.azure.com/
+    deployment: text-embedding-3-small
+    apiVersion: "2024-10-21"
+  # No API key: use Azure Workload Identity.
+```
+
+For the complete memory and embedding configuration and model requirements, see [Agent Memory](/docs/kagent/concepts/agent-memory).
+
+## Troubleshooting
+
+- **The `ModelConfig` reports `Accepted=False`:** Check whether the required `endpointFrom` ConfigMap and key exist with `kubectl describe modelconfig MODEL_CONFIG_NAME --namespace NAMESPACE`.
+- **The agent fails to become ready with a Workload Identity credential error:** Confirm the pod label and ServiceAccount annotation, and verify that the federated credential subject matches the ServiceAccount used by the pod.
+- **Foundry returns `401 Unauthorized` or `403 Forbidden`:** Confirm that the managed identity has the `Cognitive Services User` role on the Foundry resource, or that the configured API key has access to the resource.

docs-site/content/kagent/supported-providers/azure-openai.md

@@ -1,41 +1,69 @@
 ---
 title: Azure OpenAI
 description: Learn how to configure Azure OpenAI models in kagent.
-weight: 3
+weight: 3.5
 author: kagent.dev
 ---
 
 ## Configuring Azure OpenAI
 
-1. Create a Kubernetes Secret that stores the API key, replace `<your_api_key>` with an actual API key:
+The following steps use API key authentication.
 
-```shell
-export AZURE_OPENAI_API_KEY=<your_api_key>
-kubectl create secret generic kagent-azureopenai -n kagent --from-literal AZURE_OPENAI_API_KEY=$AZURE_OPENAI_API_KEY
+1. Create a Kubernetes Secret that contains your Azure OpenAI API key.
+
+```bash
+export AZURE_OPENAI_API_KEY="<your-azure-openai-api-key>"
+
+kubectl create secret generic azure-openai-api-key \
+  --namespace kagent \
+  --from-literal=api-key="${AZURE_OPENAI_API_KEY}"
 ```
 
-2. Create a ModelConfig resource that references the secret and key name, and specify the additional information that's required for the Azure OpenAI - that's the deployment name, version and the Azure AD token. You can get these values from Azure.
+2. Create a `ModelConfig` for your Azure OpenAI deployment.
 
 ```yaml
 apiVersion: kagent.dev/v1alpha2
 kind: ModelConfig
 metadata:
-  name: azuredefault-model-config
+  name: azure-openai
   namespace: kagent
 spec:
-  apiKeySecret: kagent-azureopenai
-  apiKeySecretKey: AZURE_OPENAI_API_KEY
+  apiKeySecret: azure-openai-api-key
+  apiKeySecretKey: api-key
   model: gpt-4o-mini
   provider: AzureOpenAI
   azureOpenAI:
-    azureEndpoint: "https://{yourendpointname}.openai.azure.com/"
+    azureEndpoint: "https://<account>.openai.azure.com/"
     apiVersion: "2025-03-01-preview"
-    azureDeployment: "gpt-4o-mini"
-    azureAdToken: <azure_ad_token_value>
+    azureDeployment: gpt-4o-mini
 ```
 
 For Azure OpenAI's standard models, kagent automatically configures the appropriate model capabilities.
 
-3. Apply the above resource to the cluster.
+3. Save the manifest as `azure-openai.yaml`, then apply it to your cluster.
+
+```bash
+kubectl apply -f azure-openai.yaml
+```
+
+After you apply the `ModelConfig`, you can select it from the **Model** dropdown when you create or update an agent in the kagent UI.
+
+## Authentication
+
+| Configuration | Credential |
+| --- | --- |
+| `apiKeySecret` is set | API key from the referenced Secret. |
+| `apiKeyPassthrough: true` | Azure OpenAI API key supplied as the bearer token in the caller's A2A request. This is not Microsoft Entra ID authentication. |
+| Neither field is set | Azure Workload Identity. |
+
+### Workload Identity
+
+> **Note:** Azure Workload Identity requires the Go agent runtime (`runtime: go`).
+
+Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime tries to authenticate using your Azure CLI login.
+
+> **Note:** The runtime validates Azure credentials at startup. If credentials cannot be resolved, the agent does not become ready.
+
+Follow the [AKS Workload Identity deployment guide](https://learn.microsoft.com/azure/aks/workload-identity-deploy-cluster) and grant the managed identity the `Cognitive Services User` role.
 
-Once the resource is applied, you can select the model from the Model dropdown in the UI when creating or updating agents.
+The kagent pod and ServiceAccount settings are provider-independent. Follow [Configure the agent for Azure Workload Identity](/docs/kagent/supported-providers/azure-ai-foundry#configure-the-agent-for-azure-workload-identity) and set the agent's `modelConfig` to `azure-openai`.

docs-site/content/kagent/supported-providers/byo-openai.md

@@ -51,6 +51,45 @@ You can bring your own model from an [OpenAI API-compatible](https://platform.op
 
 Good job! You added a model to kagent. Next, you can [create or update an agent](https://kagent.dev/docs/kagent/getting-started/first-agent) to use this model.
 
+## Self-hosted vLLM behind an OpenAI-compatible gateway
+
+A common self-hosted pattern places an OpenAI-compatible gateway such as [Bifrost](https://github.com/maximhq/bifrost) or [LiteLLM](https://docs.litellm.ai/) in front of a [vLLM](https://docs.vllm.ai/) server (kagent → gateway → vLLM). Configure the gateway as an OpenAI-compatible provider, exactly as shown above, with two extra things to get right.
+
+### Enable tool calling in vLLM
+
+kagent sends a `tools` array with `tool_choice: "auto"` on every request. kagent's runtime registers a built-in `ask_user` tool on every agent, so a `tools` array is always sent, even when you configure no tools yourself. The vLLM backend **must** be launched with automatic tool choice enabled, or every agent turn fails.
+
+```bash
+vllm serve Qwen/Qwen2.5-7B-Instruct \
+  --enable-auto-tool-choice \
+  --tool-call-parser hermes
+```
+
+The correct `--tool-call-parser` depends on your model family. For example, Qwen2.5 uses `hermes` and Llama 3.1 uses `llama3_json`. Parser names change across vLLM releases, so check the [vLLM tool calling docs](https://docs.vllm.ai/en/latest/features/tool_calling.html) for your model's current parser name.
+
+### Use the gateway's model identifier
+
+Set `spec.model` to the identifier your gateway routes on (often provider-prefixed, such as `vllm/Qwen/Qwen2.5-7B-Instruct`), which can differ from the bare model name vLLM serves internally. Point `openAI.baseUrl` at the gateway (LiteLLM defaults to port `4000`, Bifrost to `8080`).
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: ModelConfig
+metadata:
+  name: qwen-vllm-via-gateway
+  namespace: kagent
+spec:
+  apiKeySecret: kagent-my-provider
+  apiKeySecretKey: ${PROVIDER_API_KEY}
+  provider: OpenAI
+  model: vllm/Qwen/Qwen2.5-7B-Instruct
+  openAI:
+    baseUrl: http://litellm.kagent.svc.cluster.local:4000/v1
+```
+
+### Troubleshooting: `provider API error (status 400)`
+
+If every agent message fails with a generic `status 400`, the most common cause is that vLLM started without `--enable-auto-tool-choice` and a matching `--tool-call-parser`. Because kagent always sends `tool_choice: "auto"`, vLLM rejects the request until automatic tool choice is enabled. Restart vLLM with the flags above and retry.
+
 ## TLS Configuration
 
 To secure communication to LLMs with your own custom certificates, configure the TLS CA details in the `ModelConfig`. Then, your agents communicate with the LLM with those custom certificates. This feature is useful for internal or company-managed LLM servers.