From e7d94ae68cbfe85c745f70958f1a7f749b8c0ce2 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Fri, 7 Aug 2026 16:47:06 -0700 Subject: [PATCH 1/8] Add guardrails doc --- content/ngf/how-to/f5-ai-guardrails.md | 331 +++++++++++++++++++++++++ 1 file changed, 331 insertions(+) create mode 100644 content/ngf/how-to/f5-ai-guardrails.md diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md new file mode 100644 index 000000000..d4db74396 --- /dev/null +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -0,0 +1,331 @@ +--- +title: Secure LLM traffic with F5 AI Guardrails +weight: 900 +toc: true +f5-content-type: how-to +f5-product: FABRIC +f5-keywords: NGINX Gateway Fabric, F5 AI Guardrails, AI Guardrails, PayloadProcessor, LLM, large language model, Gateway API, Kubernetes, content policy, PII, ai-guardrails module, guardrails +f5-description: How to deploy F5 AI Guardrails with NGINX Gateway Fabric using the PayloadProcessor policy to inspect and block LLM request and response payloads. +f5-summary: > + Deploy a large language model (LLM) behind NGINX Gateway Fabric, attach a PayloadProcessor policy + that routes request and response payloads through an external Guardrails API, and verify that + disallowed content is blocked before it reaches the model or the client. +--- + +Learn how to use NGINX Gateway Fabric with F5 AI Guardrails to inspect large language model (LLM) traffic and block disallowed content before it reaches the model or the client. + +## Overview + +F5 AI Guardrails has the ability to inspect LLM traffic on two independent paths: + +- **Request path** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. +- **Response path** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. + +This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. + +## Before you begin + +You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. + +To enable the `PayloadProcessor` policy, [install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with these modifications: + +- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. +- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: + +```yaml +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors + verbs: + - get + - list + - watch +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors/status + verbs: + - update +``` + +## Deploy an LLM backend + +If you have an existing in-cluster LLM which can be queried you can skip this section. + +The following example uses the [vLLM simulator](https://github.com/llm-d/llm-d-inference-sim/tree/main), which serves canned responses from a dataset rather than running a real model, making it suitable for test and development environments. The simulator loads its dataset from a ConfigMap. Download the dataset file, then create the ConfigMap from it: + +```shell +curl -sL -o inference-sim-dataset.sqlite3 \ + https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/inference-sim-dataset.sqlite3 + +kubectl create configmap inference-sim-dataset \ + --from-file=inference-sim-dataset.sqlite3=./inference-sim-dataset.sqlite3 +``` + +{{< call-out "note" >}} +The dataset file alongside more details of the setup can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. +{{< /call-out >}} + +Deploy the LLM Deployment and Service: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/llm.yaml +``` + +Confirm the Pod is `Running`: + +```shell +kubectl get pods -l app=vllm-qwen3-32b +``` + +## Create a Gateway + +```yaml +kubectl apply -f - < +``` + +## Create an HTTPRoute + +If you are using your own LLM, change the `backendRefs.name` and `backendRefs.port` to match the LLM's Service. + +```yaml +kubectl apply -f - <" +EOF +``` + +## Configure the Guardrails backend Service + +The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: + +| Backend location | Service type | Resolved URL | +| ---------------- | ------------ | ------------ | +| External | `ExternalName` | `https://:` | +| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | + +{{< call-out "note" >}} +The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`). +{{< /call-out >}} + +For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: + +```yaml +kubectl apply -f - < + ports: + - name: https + port: 443 + protocol: TCP +EOF +``` + +For an in-cluster backend, your Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. + +{{< call-out "important" >}} +When using an `ExternalName` Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: + +```yaml +apiVersion: gateway.nginx.org/v1alpha2 +kind: NginxProxy +metadata: + name: guardrails-nginx-config +spec: + dnsResolver: + addresses: + - type: IPAddress + value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) +``` + +Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. +{{< /call-out >}} + +## Attach the PayloadProcessor policy + +Attach the `PayloadProcessor` policy to the HTTPRoute. The `extProcess.backendRef` points at the Guardrails backend Service (using the explicit port), and `authTokenRef` points at the token Secret: + +```yaml +kubectl apply -f - <}} +`PayloadProcessor` is an inherited policy. To apply guardrails to every route attached to a Gateway, set `targetRef` to `kind: Gateway`. When both a Gateway-targeted and an HTTPRoute-targeted policy apply to the same traffic, the more specific HTTPRoute-targeted policy takes precedence. +{{< /call-out >}} + +Confirm the policy was accepted: + +```shell +kubectl get payloadprocessor llm-guardrails -o yaml +``` + +The status conditions should report `Accepted=True`. A rejected policy reports `Accepted=False`; see [Troubleshooting](#troubleshooting) for common causes. + +## Send traffic + +{{< call-out "note" >}} +Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above. +{{< /call-out >}} + +All commands target `/v1/completions` on the Gateway. + +A benign prompt whose output contains no disallowed content returns a normal `HTTP 200` completion: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"What is NGINX?"}' +``` + +If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"My SSN is 123-45-6789"}' +``` + +```text +HTTP/1.1 403 Forbidden +Content-Type: application/json + +{"error":{"type":"invalid_request_error","code":"content_policy_violation", ...}} +``` + +If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"Give me a test SSN"}' +``` + +```text +HTTP/1.1 403 Forbidden +Content-Type: application/json + +{"error":{"type":"api_error","code":"content_policy_violation", ...}} +``` + +For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the Nginx Gateway Fabric repository. + +## Troubleshooting + +The `PayloadProcessor` is marked `Accepted=False` when its references cannot be resolved: + +| Condition | Cause | Fix | +| --------- | ----- | --- | +| `backend Service ... not found` | `backendRef.name`/`namespace` does not match a Service. | Apply the Guardrails backend Service; check name and namespace. | +| `ExternalName service has empty ... externalName` | `ExternalName` Service with a blank `externalName`. | Set `spec.externalName`. | +| `auth token Secret ... not found` | `authTokenRef` set but Secret missing. | Apply the token Secret, or remove `authTokenRef`. | +| NGINX error `no resolver defined to resolve `, or guardrails requests fail against an `ExternalName` backend | No `dnsResolver` configured on the NginxProxy. | Add the `dnsResolver` block and wire it via `parametersRef`. | + +## Further reading + +- [F5 AI Guardrails Documentation](https://docs.aisecurity.f5.com/) +- [Installation]({{< ref "/ngf/install/" >}}): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled. +- [Custom policies]({{< ref "/ngf/overview/custom-policies.md" >}}): learn how inherited policies attach to Gateway API resources. +- [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails): for more information on the example used in this guide. From 5fcfe96b2395311f5c8705fb0090e12026355089 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Fri, 7 Aug 2026 16:48:36 -0700 Subject: [PATCH 2/8] Use document IP address --- content/ngf/how-to/f5-ai-guardrails.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index d4db74396..743a7b487 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -106,7 +106,7 @@ kubectl describe gateways.gateway.networking.k8s.io inference-gateway Status: Addresses: Type: IPAddress - Value: 10.96.36.219 + Value: 192.0.2.0 Conditions: Message: The Gateway is accepted Reason: Accepted From e6ebd7da937beb70d31a87979e66288c9a489051 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Mon, 10 Aug 2026 10:53:34 -0700 Subject: [PATCH 3/8] Add suggested fixes --- content/ngf/how-to/f5-ai-guardrails.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 743a7b487..8cad214cf 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -1,11 +1,11 @@ --- title: Secure LLM traffic with F5 AI Guardrails +description: Deploy F5 AI Guardrails with NGINX Gateway Fabric using PayloadProcessor to inspect and block LLM traffic weight: 900 toc: true f5-content-type: how-to -f5-product: FABRIC +f5-product: F5 NGINX Gateway Fabric f5-keywords: NGINX Gateway Fabric, F5 AI Guardrails, AI Guardrails, PayloadProcessor, LLM, large language model, Gateway API, Kubernetes, content policy, PII, ai-guardrails module, guardrails -f5-description: How to deploy F5 AI Guardrails with NGINX Gateway Fabric using the PayloadProcessor policy to inspect and block LLM request and response payloads. f5-summary: > Deploy a large language model (LLM) behind NGINX Gateway Fabric, attach a PayloadProcessor policy that routes request and response payloads through an external Guardrails API, and verify that @@ -16,10 +16,10 @@ Learn how to use NGINX Gateway Fabric with F5 AI Guardrails to inspect large lan ## Overview -F5 AI Guardrails has the ability to inspect LLM traffic on two independent paths: +F5 AI Guardrails can inspect LLM traffic on two independent paths: -- **Request path** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. -- **Response path** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. +- **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. +- **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. @@ -204,10 +204,10 @@ spec: EOF ``` -For an in-cluster backend, your Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. +For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. {{< call-out "important" >}} -When using an `ExternalName` Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: ```yaml apiVersion: gateway.nginx.org/v1alpha2 @@ -310,7 +310,7 @@ Content-Type: application/json {"error":{"type":"api_error","code":"content_policy_violation", ...}} ``` -For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the Nginx Gateway Fabric repository. +For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository. ## Troubleshooting From 4fd0da3f8bc5155a7d31179ba6094c74bd97afd9 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Mon, 10 Aug 2026 22:08:32 -0700 Subject: [PATCH 4/8] Small fix to further reading link --- content/ngf/how-to/f5-ai-guardrails.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 8cad214cf..9ac744851 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -325,7 +325,7 @@ The `PayloadProcessor` is marked `Accepted=False` when its references cannot be ## Further reading -- [F5 AI Guardrails Documentation](https://docs.aisecurity.f5.com/) +- [Scan streaming in AI Security](https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html) - [Installation]({{< ref "/ngf/install/" >}}): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled. - [Custom policies]({{< ref "/ngf/overview/custom-policies.md" >}}): learn how inherited policies attach to Gateway API resources. - [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails): for more information on the example used in this guide. From fdaaee1cc3a80d436f9e5dfc8c0ba8d96ec6e2dc Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 10:41:20 -0700 Subject: [PATCH 5/8] Add various outputs for shell commands --- content/ngf/how-to/f5-ai-guardrails.md | 73 ++++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 5 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 9ac744851..6f3f7b788 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -73,10 +73,15 @@ Deploy the LLM Deployment and Service: kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/llm.yaml ``` -Confirm the Pod is `Running`: +Confirm the Pod is `Ready`: ```shell -kubectl get pods -l app=vllm-qwen3-32b +kubectl get deployment vllm-qwen3-32b +``` + +```text +NAME READY UP-TO-DATE AVAILABLE AGE +vllm-qwen3-32b 1/1 1 1 6m13s ``` ## Create a Gateway @@ -157,6 +162,23 @@ Confirm that the HTTPRoute status conditions include `Accepted=True` and `Resolv kubectl describe httproute llm-route ``` +```text +Conditions: + Last Transition Time: 2026-08-11T17:29:19Z + Message: The Route is accepted + Observed Generation: 1 + Reason: Accepted + Status: True + Type: Accepted + Last Transition Time: 2026-08-11T17:29:19Z + Message: All references are resolved + Observed Generation: 1 + Reason: ResolvedRefs + Status: True + Type: ResolvedRefs + Controller Name: gateway.nginx.org/nginx-gateway-controller +``` + ## Create the authentication token Secret Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: @@ -259,11 +281,28 @@ EOF Confirm the policy was accepted: ```shell -kubectl get payloadprocessor llm-guardrails -o yaml +kubectl describe payloadprocessor llm-guardrails ``` The status conditions should report `Accepted=True`. A rejected policy reports `Accepted=False`; see [Troubleshooting](#troubleshooting) for common causes. +```text +Conditions: + Last Transition Time: 2026-08-11T17:32:52Z + Message: The Policy is accepted + Observed Generation: 1 + Reason: Accepted + Status: True + Type: Accepted + Last Transition Time: 2026-08-11T17:32:52Z + Message: Policy is programmed in the data plane + Observed Generation: 1 + Reason: Programmed + Status: True + Type: Programmed + Controller Name: gateway.nginx.org/nginx-gateway-controller +``` + ## Send traffic {{< call-out "note" >}} @@ -280,6 +319,20 @@ curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT:$GW_IP http://:$GW_PORT:$GW_IP http://}}/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository. From 9c39ea1d87b1853d8a4f626cd5a47f397258cade Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 11:11:27 -0700 Subject: [PATCH 6/8] Refactor ordering of sections --- content/ngf/how-to/f5-ai-guardrails.md | 180 +++++++++++++------------ 1 file changed, 91 insertions(+), 89 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 6f3f7b788..558e9f378 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -27,28 +27,6 @@ This behavior is provided by the `PayloadProcessor` policy, an [inherited policy You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. -To enable the `PayloadProcessor` policy, [install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with these modifications: - -- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. -- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: - -```yaml -- apiGroups: - - gateway.nginx.org - resources: - - payloadprocessors - verbs: - - get - - list - - watch -- apiGroups: - - gateway.nginx.org - resources: - - payloadprocessors/status - verbs: - - update -``` - ## Deploy an LLM backend If you have an existing in-cluster LLM which can be queried you can skip this section. @@ -84,6 +62,97 @@ NAME READY UP-TO-DATE AVAILABLE AGE vllm-qwen3-32b 1/1 1 1 6m13s ``` +## Create the authentication token Secret + +Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: + +```yaml +kubectl apply -f - <" +EOF +``` + +## Configure the Guardrails backend Service + +The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: + +| Backend location | Service type | Resolved URL | +| ---------------- | ------------ | ------------ | +| External | `ExternalName` | `https://:` | +| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | + +{{< call-out "note" >}} +The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag or `clusterDomain` Helm value when deploying NGINX Gateway Fabric (default: `cluster.local`). +{{< /call-out >}} + +For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: + +```yaml +kubectl apply -f - < + ports: + - name: https + port: 443 + protocol: TCP +EOF +``` + +For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. + +{{< call-out "important" >}} +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on an new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: + +```yaml +apiVersion: gateway.nginx.org/v1alpha2 +kind: NginxProxy +metadata: + name: guardrails-nginx-config +spec: + dnsResolver: + addresses: + - type: IPAddress + value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) +``` + +Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. +{{< /call-out >}} + +## Deploy NGINX Gateway Fabric + +[Install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with the `PayloadProcessor` policy enabled: + +- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. +- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: + +```yaml +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors + verbs: + - get + - list + - watch +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors/status + verbs: + - update +``` + ## Create a Gateway ```yaml @@ -179,73 +248,6 @@ Conditions: Controller Name: gateway.nginx.org/nginx-gateway-controller ``` -## Create the authentication token Secret - -Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: - -```yaml -kubectl apply -f - <" -EOF -``` - -## Configure the Guardrails backend Service - -The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: - -| Backend location | Service type | Resolved URL | -| ---------------- | ------------ | ------------ | -| External | `ExternalName` | `https://:` | -| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | - -{{< call-out "note" >}} -The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`). -{{< /call-out >}} - -For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: - -```yaml -kubectl apply -f - < - ports: - - name: https - port: 443 - protocol: TCP -EOF -``` - -For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. - -{{< call-out "important" >}} -When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: - -```yaml -apiVersion: gateway.nginx.org/v1alpha2 -kind: NginxProxy -metadata: - name: guardrails-nginx-config -spec: - dnsResolver: - addresses: - - type: IPAddress - value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) -``` - -Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. -{{< /call-out >}} - ## Attach the PayloadProcessor policy Attach the `PayloadProcessor` policy to the HTTPRoute. The `extProcess.backendRef` points at the Guardrails backend Service (using the explicit port), and `authTokenRef` points at the token Secret: From bb956ddedf489ae16a7ad8446c255cf3960473d7 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 11:15:39 -0700 Subject: [PATCH 7/8] Add cli flag documentation --- content/ngf/reference/cli-help.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/ngf/reference/cli-help.md b/content/ngf/reference/cli-help.md index f70b108d3..d00e65ffa 100644 --- a/content/ngf/reference/cli-help.md +++ b/content/ngf/reference/cli-help.md @@ -63,6 +63,8 @@ This command runs the NGINX Gateway Fabric control plane. | _endpoint-picker-disable-tls_ | _bool_ | Disables TLS when connecting to the EndpointPicker. Set to true only for development/testing or when using a service mesh for encryption. (Default: `false`) | | _endpoint-picker-tls-skip-verify_ | _bool_ | Disables server certificate verification when connecting to the EndpointPicker, if TLS is enabled. REQUIRED: Must be true until Gateway API Inference Extension EndpointPicker supports mounting certificates. (Default `true`) | | _watch-namespaces_ | _list_ | Comma-separated list of namespaces to watch for resources. If not set, all namespaces are watched. The controller's own namespace is always included. | +| _payload-processor_ | _bool_ | Enable the PayloadProcessor API. PayloadProcessors enable declarative, ordered processing of HTTP request and response payloads by attaching to a Gateway or HTTPRoute, and are used to implement features such as Guardrails for AI workloads. | +| _cluster-domain_ | _string_ | The DNS domain of your Kubernetes cluster. | ## Sleep From 50c648371cb1698b0960f0545887752418478080 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 12:45:54 -0700 Subject: [PATCH 8/8] Add fixes for grammar --- content/ngf/how-to/f5-ai-guardrails.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 558e9f378..74a2d6e4d 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -21,7 +21,7 @@ F5 AI Guardrails can inspect LLM traffic on two independent paths: - **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. - **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. -This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. +This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which offloads NGINX traffic to F5 AI Guardrails to inspect. ## Before you begin @@ -42,7 +42,7 @@ kubectl create configmap inference-sim-dataset \ ``` {{< call-out "note" >}} -The dataset file alongside more details of the setup can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. +The dataset file, alongside more details of the setup, can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. {{< /call-out >}} Deploy the LLM Deployment and Service: @@ -112,7 +112,7 @@ EOF For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. {{< call-out "important" >}} -When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on an new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on a new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: ```yaml apiVersion: gateway.nginx.org/v1alpha2