Skip to content

CNS-139: docs: document query history and statement logging for self-managed - #38416

Draft
jubrad wants to merge 3 commits into
MaterializeInc:mainfrom
jubrad:justin/cns-139-docs-document-query-history-and-statement-logging-for-self
Draft

CNS-139: docs: document query history and statement logging for self-managed#38416
jubrad wants to merge 3 commits into
MaterializeInc:mainfrom
jubrad:justin/cns-139-docs-document-query-history-and-statement-logging-for-self

Conversation

@jubrad

@jubrad jubrad commented Aug 22, 2026

Copy link
Copy Markdown
Member

Query history is becoming available and enabled by default in self-managed deployments, but the docs said nothing about it. Self-managed operators had no guidance on the shipped sample rate, how to change it, or what raising it costs.

Linear: https://linear.app/materializeinc/issue/CNS-139/docs-document-query-history-and-statement-logging-for-self

Changes

  • New page self-managed-deployments/query-history.md: what query history is and who can see it, the two parameters that bound how much is collected, how to tune each via the Helm chart and via system parameters, and the CPU/storage cost of raising them.
  • console/monitoring.md: notes that query history is sampled and links to the new page.
  • reference/system-catalog/mz_internal.md: no longer says the max sample rate is "set by Materialize and cannot be changed by users". Now distinguishes Cloud (Materialize-controlled) from self-managed (operator-controlled).
  • transform-data/troubleshooting.md: this page claimed outright that "the sampling rate is not user-configurable", which directly contradicts the new page. Also corrected its "three days" window to 24 hours, which is what mz_recent_activity_log actually retains.
  • configuration-system-parameters.md: adds both statement-logging parameters to the parameter list.

Notes for the reviewer

Merge after #38406 (CNS-136). That PR exposes the chart values this page documents. Until it lands, the chart still passes --disable-statement-logging, so this page's "enabled by default" and the 0.1 default are true of the post-#38406 chart, not of main today. Documented values match #38406 as of its latest revision, including the statementLoggingTargetDataRate knob it added after QA review. The console side is #38407/#38408.

Two behaviors are worth a careful look, since both are easy to get wrong and I verified them against the source rather than assuming:

  1. Precedence. The chart value reaches environmentd as --system-parameter-default=..., so it is only a default. A ConfigMap or ALTER SYSTEM SET value is upserted into the catalog and wins. Undoing an override needs both removing the ConfigMap key and ALTER SYSTEM RESET: removing the key alone leaves the last synced value in the catalog, and resetting alone gets reapplied by the 1s sync loop.
  2. Sampling is not the only gate. statement_logging_target_data_rate (2071 B/s default) throttles sampled statements, so a rate of 1.0 does not record everything. Relatedly, the statement history collections are never truncated, so storage grows for the life of the instance even though the view only shows 24 hours. Both facts shape the cost section.

ALTER SYSTEM SET for this parameter requires mz_system, not just any superuser, since it is not user-modifiable. The page says so.

Testing

ci/test/lint-docs.sh locally: both hugo builds clean, htmltest reports only 2 pre-existing AWS Glue link errors also present on the base commit, and lint-docs-catalog.sh shows no .slt drift from the mz_internal.md edits.

🤖 Generated with Claude Code

jubrad added 3 commits August 21, 2026 20:48
Self-managed operators had no guidance on query history, which the Helm
chart now enables by default with a capped sample rate.

Add a Query History page under self-managed-deployments covering the
shipped default, how to tune the cap via the operator chart value and via
system parameters, and the CPU/storage cost of raising it. Because the
chart value is passed as --system-parameter-default, a ConfigMap or
ALTER SYSTEM SET value overrides it, so the page spells out that
precedence and how to undo an override.

Also cross-link from the console monitoring page, note the parameter in
the system-parameters guide, and stop mz_internal.md from implying the
max sample rate is Materialize-controlled only.

CNS-139
Address accuracy issues found reviewing the query history docs against
the source:

- Sampling is not the only gate. statement_logging_target_data_rate
  throttles logging by byte rate, so a sample rate of 1.0 does not
  record every statement.
- ALTER SYSTEM SET on statement_logging_max_sample_rate requires
  mz_system, not any superuser: the parameter is not user-modifiable.
- A chart value change needs a requested rollout, since upgrading the
  operator does not roll out instances by itself.
- The statement history collections are never truncated, so their
  storage grows for the life of the instance even though
  mz_recent_activity_log only surfaces 24 hours.
- Self-managed logs mz_system statements by default, which affects the
  cost of raising the rate.

Also fix transform-data/troubleshooting.md, which claimed the sample
rate is not user-configurable and gave a three-day window, and drop the
ConfigMap example duplicated from the query history page.

CNS-139
CNS-136 also exposes operator.args.statementLoggingTargetDataRate. It is
the byte-rate cap that actually bounds history growth on busy instances,
so document it alongside the sample rate rather than mentioning the
throttle only in passing.

CNS-139
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant