feat: add pod_sysctls operator config option#3095
Open
csy1204 wants to merge 2 commits into
Open
Conversation
Adds a new `pod_sysctls` option to KubernetesMetaConfiguration that
populates `pod.spec.securityContext.sysctls` in the operator-generated
StatefulSet pod template.
Motivation
----------
Cluster-wide mutating admission webhooks (e.g. internal platform policies
that inject TCP keepalive sysctls so connections through L4 load balancers
don't go stale) modify the StatefulSet pod template after the operator
creates it. The StatefulSet comparator in `compareStatefulSetWith` uses
`reflect.DeepEqual` on `pod.spec.securityContext`, so any field the
operator does not set (like `Sysctls`) but the webhook injects produces
a permanent diff. Every sync (`resync_period`, default 30m) then sees
"pod template security context in spec does not match the current one"
and triggers a rolling update + Patroni switchover, indefinitely.
By declaring the same sysctls list in the operator config, the
operator-generated template matches the webhook-mutated cluster state
and no spurious rolling restart is triggered.
Implementation
--------------
* `pkg/util/config/config.go`: add `PodSysctls []v1.Sysctl` to the
`Resources` config struct.
* `pkg/apis/acid.zalan.do/v1/operator_configuration_type.go`: add
matching CRD field.
* `pkg/apis/acid.zalan.do/v1/zz_generated.deepcopy.go`: handle the new
slice in DeepCopyInto.
* `pkg/apis/acid.zalan.do/v1/crds.go`,
`manifests/operatorconfiguration.crd.yaml`,
`charts/postgres-operator/crds/operatorconfigurations.yaml`: extend
the CRD openAPI schema (array of `{name, value}` objects).
* `pkg/controller/operator_config.go`: copy the CRD field into the
internal config struct.
* `pkg/cluster/k8sres.go`: in `generatePodTemplate`, set
`securityContext.Sysctls = c.OpConfig.PodSysctls` when non-empty.
* `pkg/cluster/k8sres_test.go`: new `TestPodSysctls` covering both the
configured and the empty case.
* `manifests/postgresql-operator-default-configuration.yaml`,
`charts/postgres-operator/values.yaml`,
`docs/reference/operator_parameters.md`: example usage and reference
documentation.
Notes
-----
* Available in the OperatorConfiguration CRD mode only (matches the
existing pattern for complex-typed options such as `sidecars`).
* The list is applied verbatim — the order and values must match what
the external mutator expects, or the comparator will still flag a
diff.
Signed-off-by: Sang Yeon Cho <sang-yeon.cho@navercorp.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
compareStatefulSetWithinpkg/cluster/cluster.gocompares the pod template'ssecurityContextviareflect.DeepEqual. The operator only ever populatesRunAsUser,RunAsGroup, andFSGroupon that struct (ingeneratePodTemplate), so any other field that an external actor — typically acluster-wide mutating admission webhook — injects into the StatefulSet pod
template produces a permanent diff.
This shows up most painfully with
sysctls. Many internal Kubernetes platformsinject TCP-keepalive sysctls (e.g.
net.ipv4.tcp_keepalive_time,net.ipv4.tcp_keepalive_intvl,net.ipv4.tcp_keepalive_probes) so that idleTCP connections through an L4 load balancer don't go stale silently. Postgres
clusters are an especially valuable target for such webhooks (pgbouncer pools
hold many long-idle connections that would otherwise meet RST on next send).
Once that webhook lands, every
resync_periodtick the operator sees the"pod template security context in spec does not match the current one" diff,
issues a rolling update, and performs a Patroni switchover. The cluster never
converges.
There is no operator-side way out of this today:
configKubernetesexposesspilo_runasuser/spilo_runasgroup/spilo_fsgroupandadditional_pod_capabilities, but no way to populatesysctlson the desiredtemplate.
Change
Adds a new
pod_sysctlsoption toKubernetesMetaConfiguration(mirroring theshape of
additional_pod_capabilities):When set,
generatePodTemplatepopulatespod.spec.securityContext.Sysctlsverbatim. The desired template then matches whatever a webhook would inject
(assuming the webhook is idempotent on a value already present), and the
StatefulSet comparator no longer flags a diff.
The list is applied verbatim — order and values must match the external
mutator. Empty (default) leaves
Sysctlsunset, preserving today's behavior.Files
pkg/util/config/config.go—PodSysctls []v1.SysctlinResourcespkg/apis/acid.zalan.do/v1/operator_configuration_type.go— matching CRD typepkg/apis/acid.zalan.do/v1/zz_generated.deepcopy.go— DeepCopy for the slicepkg/apis/acid.zalan.do/v1/crds.go,manifests/operatorconfiguration.crd.yaml,charts/postgres-operator/crds/operatorconfigurations.yaml— CRD openAPI schemapkg/controller/operator_config.go— CRD → internal config mappingpkg/cluster/k8sres.go— apply tosecurityContext.Sysctlspkg/cluster/k8sres_test.go—TestPodSysctls(configured + empty cases)manifests/postgresql-operator-default-configuration.yaml,charts/postgres-operator/values.yaml,docs/reference/operator_parameters.md— examples and reference docsTest
Full
./pkg/cluster/,./pkg/util/config/,./pkg/controller/,./pkg/apis/...test suites pass.Scope
Available in OperatorConfiguration CRD mode only, matching the existing pattern
for complex-typed config (e.g.
sidecarswhich is also CRD-mode only).DCO signed off.