Skip to content

Suppress successful autogrow PATCH response logs - #4537

Open
aymeric-ld wants to merge 1 commit into
CrunchyData:mainfrom
aymeric-ld:fix-autogrow-curl-log-noise
Open

Suppress successful autogrow PATCH response logs#4537
aymeric-ld wants to merge 1 commit into
CrunchyData:mainfrom
aymeric-ld:fix-autogrow-curl-log-noise

Conversation

@aymeric-ld

Copy link
Copy Markdown

Checklist:

  • Have you added an explanation of what your changes do and why you'd like them to be included?
  • Have you updated or added documentation for the change, as applicable?
  • Have you tested your changes on all related environments with successful results, as applicable?
    • Have you added automated tests?

Type of Changes:

  • New feature
  • Bug fix
  • Documentation
  • Testing enhancement
  • Other

What is the current behavior (link to any open issues here)?

In some autogrow scenarios, replication-cert-copy can generate a large amount of sidecar log output while a volume remains above the configured usage threshold. This was already raised in #4391.

In the reported environment, two replicas generated roughly 2 GiB of logs per day while the volume remained above the autogrow threshold.

The generated replication-cert-copy script checks volume usage every five seconds and PATCHes the Pod annotation when autogrow should be triggered. A successful Pod PATCH returns the updated Pod JSON, and curl writes that response body to stdout by default. While expansion is pending or blocked, that full JSON response is logged repeatedly.

While checking the reported path, I found the same successful-response logging pattern in the pgBackRest autogrow script, so this change updates both autogrow PATCH paths.

What is the new behavior (if this is a feature change)?

Successful autogrow PATCH response bodies are discarded with --output /dev/null. Curl progress output is suppressed with --silent, while curl error output is preserved with --show-error.

Autogrow behavior is unchanged: annotations are still patched, and operator events/status/PVC state remain the source of autogrow diagnostics.

This intentionally does not change the repeated PATCH behavior or autogrow decision logic. It only prevents successful Kubernetes API response bodies from being emitted to sidecar stdout.

No user-facing documentation was added because this removes unintended successful response output without changing user-facing configuration or behavior.

  • Breaking change (fix or feature that would cause existing functionality to change)

Other Information:

Tested with:

go test ./internal/postgres -run 'TestReloadCommand|TestInstancePod'
go test ./internal/pgbackrest -run 'TestReloadCommand|TestReconcile'
go test ./internal/postgres ./internal/pgbackrest
go test ./internal/controller/postgrescluster -run 'TestAddPGBackRestToInstancePodSpec|TestGenerateInstanceStatefulSetIntent'

These tests cover the generated sidecar command strings, shellcheck validation for the generated scripts, and the affected golden Pod/StatefulSet output. That is the relevant validation for this change because the PR changes the generated curl invocation only; it does not change autogrow sizing or reconciliation logic.

Autogrow sidecar scripts patch Pod annotations using curl. Successful Kubernetes PATCH responses include the full Pod JSON, which curl writes to stdout by default. Since these scripts run in containers, that response is collected as sidecar logs and can repeat while volume expansion is pending.

Discard successful response bodies while preserving curl error output. Apply the same behavior to Postgres instance and pgBackRest autogrow annotation scripts.

Issue: CrunchyData#4391
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant