CNS-139: docs: document query history and statement logging for self-managed - #38416
Draft
jubrad wants to merge 3 commits into
Draft
Conversation
Self-managed operators had no guidance on query history, which the Helm chart now enables by default with a capped sample rate. Add a Query History page under self-managed-deployments covering the shipped default, how to tune the cap via the operator chart value and via system parameters, and the CPU/storage cost of raising it. Because the chart value is passed as --system-parameter-default, a ConfigMap or ALTER SYSTEM SET value overrides it, so the page spells out that precedence and how to undo an override. Also cross-link from the console monitoring page, note the parameter in the system-parameters guide, and stop mz_internal.md from implying the max sample rate is Materialize-controlled only. CNS-139
Address accuracy issues found reviewing the query history docs against the source: - Sampling is not the only gate. statement_logging_target_data_rate throttles logging by byte rate, so a sample rate of 1.0 does not record every statement. - ALTER SYSTEM SET on statement_logging_max_sample_rate requires mz_system, not any superuser: the parameter is not user-modifiable. - A chart value change needs a requested rollout, since upgrading the operator does not roll out instances by itself. - The statement history collections are never truncated, so their storage grows for the life of the instance even though mz_recent_activity_log only surfaces 24 hours. - Self-managed logs mz_system statements by default, which affects the cost of raising the rate. Also fix transform-data/troubleshooting.md, which claimed the sample rate is not user-configurable and gave a three-day window, and drop the ConfigMap example duplicated from the query history page. CNS-139
CNS-136 also exposes operator.args.statementLoggingTargetDataRate. It is the byte-rate cap that actually bounds history growth on busy instances, so document it alongside the sample rate rather than mentioning the throttle only in passing. CNS-139
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Query history is becoming available and enabled by default in self-managed deployments, but the docs said nothing about it. Self-managed operators had no guidance on the shipped sample rate, how to change it, or what raising it costs.
Linear: https://linear.app/materializeinc/issue/CNS-139/docs-document-query-history-and-statement-logging-for-self
Changes
self-managed-deployments/query-history.md: what query history is and who can see it, the two parameters that bound how much is collected, how to tune each via the Helm chart and via system parameters, and the CPU/storage cost of raising them.console/monitoring.md: notes that query history is sampled and links to the new page.reference/system-catalog/mz_internal.md: no longer says the max sample rate is "set by Materialize and cannot be changed by users". Now distinguishes Cloud (Materialize-controlled) from self-managed (operator-controlled).transform-data/troubleshooting.md: this page claimed outright that "the sampling rate is not user-configurable", which directly contradicts the new page. Also corrected its "three days" window to 24 hours, which is whatmz_recent_activity_logactually retains.configuration-system-parameters.md: adds both statement-logging parameters to the parameter list.Notes for the reviewer
Merge after #38406 (CNS-136). That PR exposes the chart values this page documents. Until it lands, the chart still passes
--disable-statement-logging, so this page's "enabled by default" and the0.1default are true of the post-#38406 chart, not ofmaintoday. Documented values match #38406 as of its latest revision, including thestatementLoggingTargetDataRateknob it added after QA review. The console side is #38407/#38408.Two behaviors are worth a careful look, since both are easy to get wrong and I verified them against the source rather than assuming:
environmentdas--system-parameter-default=..., so it is only a default. A ConfigMap orALTER SYSTEM SETvalue is upserted into the catalog and wins. Undoing an override needs both removing the ConfigMap key andALTER SYSTEM RESET: removing the key alone leaves the last synced value in the catalog, and resetting alone gets reapplied by the 1s sync loop.statement_logging_target_data_rate(2071 B/s default) throttles sampled statements, so a rate of1.0does not record everything. Relatedly, the statement history collections are never truncated, so storage grows for the life of the instance even though the view only shows 24 hours. Both facts shape the cost section.ALTER SYSTEM SETfor this parameter requiresmz_system, not just any superuser, since it is not user-modifiable. The page says so.Testing
ci/test/lint-docs.shlocally: both hugo builds clean,htmltestreports only 2 pre-existing AWS Glue link errors also present on the base commit, andlint-docs-catalog.shshows no.sltdrift from themz_internal.mdedits.🤖 Generated with Claude Code