Ingest New Documentation

docs/Collecting Metrics/Secrets Management/Secret Stores/Vault.mdx

@@ -34,6 +34,12 @@ This page covers Vault specific setup. For the full resolver overview and syntax
 
 Netdata reads existing secrets from Vault. It does not create or renew Vault tokens. If the configured token expires or becomes invalid, secret resolution fails until Netdata can read a valid token again. If you use `token_file` mode, Netdata re-reads the file on every secret resolution, so an external process (e.g. Vault Agent, a cron job) can renew the token by writing to the file. For KV v2 secrets, Netdata does not add `/data/` to the path automatically.
 
+The Dynamic Configuration **Test** action initiates one real authenticated `GET /v1/auth/token/lookup-self` request using the configured token, namespace header, TLS settings, proxy path, and timeout. HTTP 200 is operational success by status; its response body is not inspected. An exact permission-only HTTP 403 body is reported as validation-only because it cannot reliably distinguish a valid token without self-lookup permission from an invalid token on older Vault versions or another authentication restriction. Invalid-token, ambiguous, malformed, or oversized HTTP 403 bodies fail, as does every other HTTP status.
+
+The request does not read a secret or prove access to any secret path. It also cannot prove that an intermediary preserved the namespace header or that Vault used it. A gateway or allowlist can block the self-lookup route even when configured secret reads would work. When the request reaches Vault, it creates normal audit and activity evidence. For a limited-use service token, Test can consume the final remaining use and cause later secret resolution to fail; an external proxy or service mesh can also retry independently.
+
+A configured token file must resolve to a regular file no larger than 1 MiB. Symlinks to regular files are supported.
+
 
 ## Setup
 
@@ -90,7 +96,7 @@ The following options can be defined for this secretstore backend.
 |  | [tls_skip_verify](#option-tls-skip-verify) | Disable TLS certificate verification for Vault requests. | no | no |
 |  | timeout | Timeout in seconds for HTTP requests made by this secretstore backend. | 3 | no |
 | **Token** | mode_token.token | Vault token value. Required when `mode` is `token`. |  | yes |
-| **Token File** | mode_token_file.path | Path to a file containing the Vault token. Required when `mode` is `token_file`. |  | yes |
+| **Token File** | [mode_token_file.path](#option-token-file-mode-token-file-path) | Path to a file containing the Vault token. Required when `mode` is `token_file`. |  | yes |
 
 <a id="option-mode"></a>
 ##### mode
@@ -109,6 +115,12 @@ Prefer `token_file` for production so the token is stored separately from the se
 This is insecure. Use it only as a temporary workaround or in a non-production environment.
 
 
+<a id="option-token-file-mode-token-file-path"></a>
+##### mode_token_file.path
+
+The path may be a regular file or a symlink to a regular file. The file must be no larger than 1 MiB.
+
+
 
 </details>
 
@@ -256,6 +268,13 @@ Check the Netdata Agent logs when the collector starts or restarts. Vault resolv
 Check the Vault token policy and, if you use Vault Enterprise namespaces, confirm that `namespace` is correct. If you use a short-lived token, make sure the token is renewed or replaced before it expires.
 
 
+### The Dynamic Configuration Test fails
+
+The Test action makes a real authenticated request to Vault's token self-lookup endpoint. HTTP 200 is operational success by status; its response body is not inspected. An exact permission-only HTTP 403 body is reported as validation-only because it cannot reliably prove whether the token is valid. Invalid-token, ambiguous, malformed, or oversized HTTP 403 bodies fail. An unreachable endpoint and every other HTTP status also fail.
+
+Operational success proves only that the responding endpoint returned HTTP 200 for self-lookup at that time. It does not prove that Vault accepted the token or that the token can read a particular secret path. Check secret-path policies separately. A gateway or deployment allowlist can reject the self-lookup route even when configured secret reads would work.
+
+
 ### Secret or key is not found
 
 Check the operand carefully:

docs/Collecting Metrics/Service Discovery/Discoverer/HTTP endpoint.mdx

@@ -46,8 +46,10 @@ Each discovery cycle, the discoverer:
 
 - Only **one URL per pipeline**. To pull from multiple sources, configure multiple HTTP discovery pipelines (each as its own UI entry, or split the file into one job per source).
 - **Response size** is capped at 10 MiB.
+- A DynCfg **test** performs one complete fetch only when `method` is empty/default or exactly `GET`. It uses the configured authentication, headers, TLS, proxy, redirects, and timeout; requires HTTP 200; and parses every returned item. Redirects can make up to 10 HTTP requests. The test discards the targets without evaluating `services:` rules or creating jobs. Other configured methods are intentionally validation-only.
 - **One-shot mode** (`interval: 0`) fetches a single time when the pipeline starts. It does **not** refetch on SD reload — recreate the pipeline to refresh.
-- **`bearer_token_file`** under `/var/run/secrets/` is treated as optional when Netdata is **not** running in Kubernetes (so the same config can be used in a Helm deployment without erroring out on dev hosts).
+- A missing **`bearer_token_file`** under `/var/run/secrets/` is treated as optional when Netdata is **not** running in Kubernetes (so the same config can be used in a Helm deployment without erroring out on dev hosts).
+- Configured bearer-token and TLS files have [file-safety requirements](#option-headers-username-password-bearer-token-file-proxy-url-tls-skip-verify-etc).
 - The discoverer does not introspect the items it received — anything beyond what the upstream endpoint provides must be inferred via service rules.
 
 
@@ -112,7 +114,7 @@ With `auto`, the decoder uses `Content-Type` when it is unambiguous (`applicatio
 <a id="option-headers-username-password-bearer-token-file-proxy-url-tls-skip-verify-etc"></a>
 ##### headers / username / password / bearer_token_file / proxy_url / tls_skip_verify / etc.
 
-See any go.d HTTP-based collector (`httpcheck`, `prometheus`, `nginx`, …) for the full set. Notable: when `bearer_token_file` points under `/var/run/secrets/` and Netdata is **not** running inside Kubernetes, missing token files are silently ignored.
+See any go.d HTTP-based collector (`httpcheck`, `prometheus`, `nginx`, …) for the full set. Bearer-token and TLS CA paths must resolve to regular files no larger than 1 MiB. When both `tls_cert` and `tls_key` are configured, each path has the same requirement. Symlinks to regular files are supported. When `bearer_token_file` points under `/var/run/secrets/` and Netdata is **not** running inside Kubernetes, a missing token file is silently ignored.
 
 
 
@@ -311,6 +313,13 @@ Your endpoint mixes shapes — some items target `httpcheck`, some target `prome
 
 After enabling the discoverer, confirm the endpoint is reachable and items are being parsed.
 
+### Test a UI-managed configuration
+
+DynCfg test performs one complete production fetch for an empty/default or exact `GET` method. Success proves that the request could be built, the configured auth/TLS/proxy/redirect path worked at that instant, the endpoint returned HTTP 200 with at most 10 MiB, and every item parsed into a target.
+
+The fetched targets are discarded: the test does not evaluate `services:` rules, render or validate collector jobs, publish targets, or install jobs. Non-GET methods are intentionally reported as validation-only because executing them could mutate the endpoint.
+
+
 ### Confirm the endpoint is being fetched
 
 Watch the agent log for `discoverer=http` messages. With systemd:

docs/Collecting Metrics/Service Discovery/Discoverer/Local listening processes.mdx

@@ -34,7 +34,7 @@ This page covers `net_listeners`-specific setup. For the broader Service Discove
 
 Each discovery cycle, the discoverer:
 
-1. **Reads the kernel's TCP/UDP listening-socket table** via the bundled `local-listeners` helper (which reads `/proc` on Linux, `netstat`-equivalents elsewhere).
+1. **Reads the kernel's TCP/UDP listening-socket table** via the bundled `local-listeners` helper, which reads Linux procfs socket and process data.
 2. **Builds one target per `(protocol, IP, port, process)` tuple**, exposing `.Protocol`, `.IPAddress`, `.Port`, `.Comm` (process basename), `.Cmdline` (full command line), and `.Address` (the convenience `IPAddress:Port`).
 3. **Caches** each target for 10 minutes so a brief disappearance (process restart) does not churn collector jobs.
 4. **Runs the `services:` rules** against each target. The stock conf carries ~100 curated rules covering the bulk of go.d modules (databases, web servers, caches, message queues, exporters).
@@ -45,6 +45,7 @@ Each discovery cycle, the discoverer:
 
 - Only **local** listeners are visible. Discovering services on other hosts requires another discoverer (`http`, `snmp`, `k8s`, or a custom one).
 - The discoverer needs to **read kernel socket information**. On Linux this works for processes owned by other users only when Netdata can read the appropriate `/proc/<pid>/net` files; the Netdata installer configures this via the `local-listeners` setuid helper.
+- The bundled `local-listeners` helper is currently built for Linux. A missing or non-executable helper makes both normal discovery and the DynCfg operational test fail.
 - **Containerised services in `host` networking** appear as listeners and are picked up here, not by the Docker discoverer. Services in private container networks must be discovered by the Docker discoverer instead.
 - The discoverer does not introspect process runtime — anything beyond port/`comm`/`cmdline` (e.g. config-file path, version, runtime URL prefix) must be inferred via service rules or known by convention.
 
@@ -244,6 +245,13 @@ The last rule in the stock conf catches generic Prometheus exporters by port. `p
 
 After enabling the discoverer, confirm it is finding listeners and producing jobs.
 
+### Test a candidate configuration without applying it
+
+A DynCfg `test` runs the same installed `nd-run` and `local-listeners` path once, with the production flags and configured `timeout`, then parses the complete snapshot and discards it. Success proves that this one helper invocation and parse worked on the local host at that instant.
+
+The test does not start discovery, cache or publish targets, evaluate `services:` rules, create collector jobs, or prove that a generated job can connect to its service. It requires no permissions beyond normal local-listener discovery. Work and buffered output scale with the host's socket/process inventory; caller cancellation and `timeout` bound elapsed time, not the number of inspected entries.
+
+
 ### Confirm listeners are being scanned
 
 Watch the agent log for `discoverer=net_listeners` messages. With systemd:
@@ -292,4 +300,4 @@ The stock `exporter` catch-all rule (last in the file) is greedy by design — a
 
 ### Generated jobs fail to start
 
-The discoverer creates jobs but does not run them. Common causes: the rendered template assumes credentials the local service rejects (e.g. RabbitMQ default `guest:guest`); `0.0.0.0` listeners produce `0.0.0.0:port` addresses that the collector cannot connect to (use `127.0.0.1` in the template if appropriate); the service has TLS but the template uses HTTP.
+The discoverer creates jobs but does not run them. Common causes: the rendered template assumes credentials the local service rejects (e.g. RabbitMQ default `guest:guest`); a rule renders the wrong port or protocol; or the service has TLS but the template uses HTTP.

docs/Collecting Metrics/Service Discovery/Service Discovery.mdx

@@ -18,7 +18,7 @@ Each SD pipeline is a `discoverer:` (where to look) plus a list of `services:` r
 
 ### Jump To
 
-[How it works](#how-it-works) • [Configuration file structure](#configuration-file-structure) • [Rule evaluation semantics](#rule-evaluation-semantics) • [Template helper reference](#template-helper-reference) • [config_template rendering](#config_template-rendering) • [Supported discoverers](#supported-discoverers) • [Mixing discoverers](#mixing-discoverers) • [Troubleshooting](#troubleshooting)
+[How it works](#how-it-works) • [Configuration file structure](#configuration-file-structure) • [Testing configurations](#testing-configurations) • [Rule evaluation semantics](#rule-evaluation-semantics) • [Template helper reference](#template-helper-reference) • [config_template rendering](#config_template-rendering) • [Supported discoverers](#supported-discoverers) • [Mixing discoverers](#mixing-discoverers) • [Troubleshooting](#troubleshooting)
 
 
 ## How it works
@@ -57,6 +57,16 @@ services:
 - Where each discoverer's stock conf ships (with the Netdata package, with the Helm chart, or not at all) is documented on its per-discoverer page.
 
 
+## Testing configurations
+
+Testing a UI-managed pipeline builds a complete temporary pipeline but does not publish targets, install jobs, or make the configuration persistent. The response says explicitly when only configuration validation was possible.
+
+Operational guarantees depend on the discoverer. Docker runs one bounded container-list query. Local-listener discovery runs and parses one helper snapshot. HTTP discovery runs one complete production fetch only when `method` is empty/default or exactly `GET`; it uses the configured authentication, headers, TLS, proxy, redirects, and timeout, requires HTTP 200, enforces the 10 MiB response limit, and parses every returned item. Redirect handling can issue at most 10 requests.
+
+The HTTP test discards the parsed targets. It does not evaluate `services:` rules, render or validate collector jobs, publish targets, or install jobs. A non-empty HTTP method other than exact `GET`, plus Kubernetes and SNMP, is intentionally validation-only.
+
+
+
 ## Rule evaluation semantics
 
 For each discovered target, the engine walks the `services:` array top-to-bottom. The two rule shapes behave differently:

docs/Network Performance Monitoring/Network Flows/Configuration.mdx

@@ -190,13 +190,15 @@ Per-tier values:
 
 | Key | Default per tier | Notes |
 |---|---|---|
-| `size_of_journal_files` | `10GB` | Disk budget for this tier. Minimum `100MB`. Set to `null` to disable size-based retention on this tier. |
+| `size_of_journal_files` | `10GB` | Retained-artifact budget for journal data and finalized per-journal facet sidecars. Minimum `100MB`. Set to `null` to disable size-based retention on this tier. |
 | `duration_of_journal_files` | `null` | Optional time budget for this tier. The default disables time-based eviction; set a duration such as `24h` or `14d` to add an age cap. |
 
 Either configured limit can evict old files. The tier expires whichever configured limit is hit first. At least one of the two must be set per tier (validation enforces this).
 
 If you omit a tier entry entirely, that tier uses the built-in defaults (`10GB` / no time limit). If you provide a tier entry but omit one of the two knobs, the omitted knob falls back to its built-in default. Setting either to `null` explicitly disables that limit on that tier.
 
+The size budget is not a strict filesystem quota. The active journal counts toward the budget but is protected from deletion, so a tier can temporarily exceed its limit until rotation. Shared `facet-state.bin`, temporary sidecar files, and unrelated files are not charged to an individual tier.
+
 Standalone CLI runs still accept the legacy uniform retention flags:
 `--netflow-retention-size-of-journal-files` and
 `--netflow-retention-duration-of-journal-files`. They apply the same value to