Self-hosting: using Kubernetes Secrets instead of hardcoded values is not working

[TheRealFloatDev]

Provide environment information

Not applicable

Describe the bug

According to the Kubernetes Documentation there are two ways to provide secrets.

1. Hardcoding them like so:

secrets:
  enabled: true
  sessionSecret: "your-32-char-hex-secret-1"
  magicLinkSecret: "your-32-char-hex-secret-2"
  # ...

2. Using a Kubernetes Secret like so:

# Recommended: existingSecret, must contain at least the following keys:
# - SESSION_SECRET
# - MAGIC_LINK_SECRET
# - ENCRYPTION_KEY
# - MANAGED_WORKER_SECRET
# - OBJECT_STORE_ACCESS_KEY_ID
# - OBJECT_STORE_SECRET_ACCESS_KEY
secrets:
  enabled: false
  existingSecret: "your-existing-secret"

Sadly the second version is simply not working.
If you set the secret this way they are not recognized leaving you with a bunch of errors like this one (when starting the web app):

+ NODE_PATH=/triggerdotdev/node_modules/.pnpm/node_modules exec dumb-init node --max-old-space-size=8192 ./build/server.js
/triggerdotdev/node_modules/.pnpm/zod@3.25.76/node_modules/zod/v3/types.cjs:120
        throw result.error;
        ^
ZodError: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "SESSION_SECRET"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "MAGIC_LINK_SECRET"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "ENCRYPTION_KEY"
    ],
    "message": "Required"
  }
]

Reproduction repo

https://trigger.dev/docs/self-hosting/kubernetes#custom-values

To reproduce

1. Get yourself the example values.yaml from the docs
2. set secrets.enable to false to not use the hardcoded secrets
3. Create a secret and reference it in secrets.existingSecret
4. Deploy the helm chart using your values file
5. Encounter deployment issues due to seemingly missing secrets

Additional information

No response

[matt-aitken]

@TheRealFloatDev can you provide some version numbers and more details about your setup and where you're deploying to?

We have a lot of self-hosters using this setup.

[TheRealFloatDev]

Sure.

Regarding "where" I am deploying to:
I am deploying to a RKE2 Kubernetes Cluster, running Hetzner Cloud AMD64 servers.

Regarding the version
I am using the docs-provided install command which automatically uses the latest version of 4.0.

helm upgrade -n trigger --install trigger \
  oci://ghcr.io/triggerdotdev/charts/trigger \
  --version "~4.0.0" \
  -f trigger-production-values.yaml

The version it installed is v4.0.5.

Exact content of the trigger-production-values.yaml

# REQUIRED: Generate your own secrets using: openssl rand -hex 16
secrets:
  enabled: false # False to use existing secret - true to specify them here directly (only for testing/dev)
  existingSecret: "trigger-secret" # Name of the Kubernetes secret created from trigger-secret.yaml

# Production webapp configuration
webapp:
  # Origin configuration
  appOrigin: "REDACTED" # Not redacted in the actual file
  loginOrigin: "REDACTED" # Not redacted in the actual file
  apiOrigin: "REDACTED" # Not redacted in the actual file

  # Production ingress
  ingress:
    enabled: true
    className: "nginx"
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    hosts:
      - host: "REDACTED" # Not redacted in the actual file
        paths:
          - path: /
            pathType: Prefix
    tls:
      - secretName: trigger-tls
        hosts:
          - "REDACTED" # Not redacted in the actual file

  resources:
    limits:
      cpu: 2000m
      memory: 4Gi
    requests:
      cpu: 500m
      memory: 1Gi

# Production PostgreSQL (or use external)
postgres:
  primary:
    persistence:
      enabled: true
      size: 20Gi # Keep an eye on this depending on your usage
      storageClass: "longhorn"
    resources:
      limits:
        cpu: 1000m
        memory: 2Gi
      requests:
        cpu: 500m
        memory: 1Gi

# Production Redis (or use external)
redis:
  master:
    persistence:
      enabled: true
      size: 10Gi # Keep an eye on this depending on your usage
      storageClass: "longhorn"
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 250m
        memory: 512Mi

# Production ClickHouse
clickhouse:
  # Set to true to enable TLS/secure connections in production
  secure: false
  persistence:
    enabled: true
    size: 10Gi # Keep an eye on this depending on your usage
    storageClass: "longhorn"
  # ClickHouse can be very resource intensive, so we recommend setting limits and requests accordingly
  # Note: not doing this can cause OOM crashes which will cause issues across many different
  resources:
    limits:
      cpu: 4000m
      memory: 16Gi
    requests:
      cpu: 2000m
      memory: 8Gi

# Production S3-compatible object storage
minio:
  deploy: false # Disable MinIO deployment in production
s3:
  deploy: false # Disable built-in S3 deployment in production (should be the minio setting above, but just making sure)
  external:
    endpoint: "REDACTED" # Not redacted in the actual file
    existingSecret: "trigger-secret" # Use the same secret created from trigger-secret.yaml (contains OBJECT_STORE_ACCESS_KEY_ID and OBJECT_STORE_SECRET_ACCESS_KEY)
    existingSecretAccessKeyIdKey: "OBJECT_STORE_ACCESS_KEY_ID"
    existingSecretSecretAccessKeyKey: "OBJECT_STORE_SECRET_ACCESS_KEY"

# Production Registry
registry:
  deploy: true # Use the built-in Docker registry for storing images
  auth:
    username: "registry-user"
    password: "your-strong-registry-password"
  persistence:
    enabled: true
    size: 45Gi # Keep an eye on this depending on your usage
    storageClass: "longhorn"

  # Production ingress
  ingress:
    enabled: true
    className: "nginx"
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    hosts:
      - host: "REDACTED" # Not redacted in the actual file
        paths:
          - path: /
            pathType: Prefix
    tls:
      - secretName: registry-tls
        hosts:
          - "REDACTED" # Not redacted in the actual file

# Production Supervisor (Kubernetes worker orchestrator)
supervisor:
  resources:
    limits:
      cpu: 500m
      memory: 1Gi
    requests:
      cpu: 250m
      memory: 512Mi

# Telemetry configuration
telemetry:
  enabled: false # Disable telemetry in production

I created this file based on the following resources:

• https://trigger.dev/docs/self-hosting/kubernetes
• https://github.com/triggerdotdev/trigger.dev/blob/main/hosting/k8s/helm/values-production-example.yaml
• https://github.com/triggerdotdev/trigger.dev/blob/main/hosting/k8s/helm/values.yaml

Secret is created like this

apiVersion: v1
kind: Secret
metadata:
  name: trigger-secret
  namespace: trigger
stringData:
  # Generate secrets using `openssl rand -hex 16`
  SESSION_SECRET:
  MAGIC_LINK_SECRET:
  ENCRYPTION_KEY:
  MANAGED_WORKER_SECRET:
  OBJECT_STORE_ACCESS_KEY_ID:
  OBJECT_STORE_SECRET_ACCESS_KEY:

Of course with the stringData actually populated

[nicktrn]

@claude I think this should be working, please check why it might not be

[claude]

Claude finished @nicktrn's task in 2m 9s —— View job • claude/issue-2859-20260109-1126 • Create PR ➔

---

Fixed: Support secrets.existingSecret for Core Secrets

• Examine Helm chart templates to understand how secrets.existingSecret is used
• Check if environment variables are properly populated from the existing secret
• Identify the root cause of why secrets aren't being recognized
• Implement the fix

Root Cause

The bug was in webapp.yaml and supervisor.yaml. The templates only checked {{- if .Values.secrets.enabled }} before populating environment variables from secrets. When users set secrets.enabled: false to use existingSecret, the critical environment variables (SESSION_SECRET, MAGIC_LINK_SECRET, ENCRYPTION_KEY, MANAGED_WORKER_SECRET) were never set.

Fix Applied

Changed the condition from:

{{- if .Values.secrets.enabled }}

To:

{{- if or .Values.secrets.enabled .Values.secrets.existingSecret }}

This allows the environment variables to be populated from either:

1. The chart-generated secret (when secrets.enabled: true)
2. A user-provided existing secret (when secrets.existingSecret is set)

The trigger-v4.secretsName helper already correctly returns the existingSecret name when provided.

Files Changed

• hosting/k8s/helm/templates/webapp.yaml - Fixed condition for SESSION_SECRET, MAGIC_LINK_SECRET, ENCRYPTION_KEY, MANAGED_WORKER_SECRET
• hosting/k8s/helm/templates/supervisor.yaml - Fixed condition for MANAGED_WORKER_SECRET

---

|

[nicktrn]

hey @TheRealFloatDev there's a helm prerelease here, any chance you could check this fixes it? will merge when you give the go ahead!

#2860 (comment)

🧭 Helm Chart Prerelease Published

Version: 4.0.5-pr2860.012b943

Install:

helm upgrade --install trigger \
  oci://ghcr.io/triggerdotdev/charts/trigger \
  --version "4.0.5-pr2860.012b943"

⚠️ This is a prerelease for testing. Do not use in production.