chart: allow a gateway-only resources override (MLI-8534)

[jay-tervala]

Linear: MLI-8534

Problem

.Values.resources is rendered into all three model-engine deployments:

templates/gateway_deployment.yaml:80
templates/cacher_deployment.yaml:67
templates/endpoint_builder_deployment.yaml:79

Their memory profiles aren't comparable. On ml-serving-new right now:

Component	Pods	Actual memory
gateway	100	median 28.8 GiB, p90 32.1 GiB, peak 103 GB
cacher	1	1.0 GiB
endpoint-builder	2	0.7 GiB each

The gateway holds async-task results in memory while proxying them inline (#868's rate-limit comments describe the same payload path). The other two don't.

With one shared block the only options are under-declaring the gateway or over-reserving for the other two. Today it's the former: the gateway declares no memory request at all, so the scheduler treats those nodes as memory-empty and packs up to 3 gateway pods onto a 120.3 GiB node. On 2026-08-25 that produced 46 OOMKills across the fleet, node-level rather than container-scoped — so the node OOM killer picks its own victim, which may be an unrelated workload.

Change

Adds an optional gateway.resources, alongside the existing gateway.* tuning keys from #867/#868. When unset, the gateway falls back to .Values.resources exactly as today.

Backwards compatibility

Verified with helm template against the live prod values:

• gateway.resources unset — gateway, cacher and builder all render {cpu: 2, ephemeral-storage: 256Mi}. Byte-identical to current output, so this is a no-op for every existing consumer including SGP.
• gateway.resources set — gateway picks it up; cacher and builder are untouched.

helm lint clean.

Not in this PR

No values are changed. The sizing (a memory request that matches reality, and whether a limit ships at all) lands separately in the deploy values, and is still under discussion — see MLI-8534 for why a limit is harder than it looks while the gateway's steady state sits at ~29 GiB and the HPA is pinned at maxReplicas: 100.

One question for review

I bumped Chart.yaml to 0.2.10, since the templates changed.

Worth flagging that the published 0.2.9 in public.ecr.aws/b2z8n5q1 does not match 0.2.9 at current main — #868's sidecarCPURequest and readinessProbeTimeoutSeconds are in source but not in the published tarball, under the same version string. So a bump here means the next publish also ships those. That seemed right to me, but happy to drop the bump if you'd rather sequence it separately.

🤖 Generated with Claude Code

Greptile Summary

The chart adds gateway-specific resource overrides while retaining shared resource defaults, and makes the gateway VPA’s controlled resources configurable.

• Deep-merges gateway.resources over the shared resource block without mutating values used by the cacher or endpoint builder.
• Allows operators to stop the gateway VPA from controlling memory when memory requests are pinned.
• Advances the model-engine chart version to 0.2.10.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
charts/model-engine/Chart.yaml	Advances the chart version from 0.2.9 to 0.2.10 for the template changes.
charts/model-engine/templates/gateway_deployment.yaml	Deep-merges gateway-only resources over shared defaults while protecting the shared values from mutation.
charts/model-engine/templates/gateway_vpa.yaml	Renders a configurable gateway VPA controlled-resource list with the existing CPU-and-memory default.
charts/model-engine/values.yaml	Documents and supplies defaults for gateway-specific resources and VPA control.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Shared["resources"] --> Merge["Deep merge"]
  GatewayOverride["gateway.resources"] --> Merge
  Merge --> Gateway["Gateway resources"]
  Shared --> Cacher["Cacher resources"]
  Shared --> Builder["Endpoint-builder resources"]
  VPAConfig["gateway.vpaControlledResources"] --> GatewayVPA["Gateway VPA policy"]

Loading

_{Reviews (3): Last reviewed commit: "chore: empty commit to retrigger CI" | Re-trigger Greptile}

Context used:

• Knowledge Base — Deployment and operational configuration
• Knowledge Base — Helm release and Kubernetes startup watcher

[lorenzo-norcini-scale]

Does this do deep-merge gateway.resources with .Values.resources? Does a memory-only override replace also cpu and ephemeral storage?

[lorenzo-norcini-scale]

Prod enables the gateway VPA in Initial mode with memory controlled and capped at 8Gi. Does this conflict with 32GB and what's the behavior?