Support ref+<backend>:// secret resolution in environment values (vals/helmfile URI syntax)

[johan-nilsson-82]

Description

When running services that need secrets (database passwords, API keys, etc.), there's no native way to resolve them from external secret stores like OpenBao, HashiCorp Vault, AWS Secrets Manager, or others. Users currently rely on external preprocessors:

vals eval -f compose.yaml | docker compose -f - up

or shell variable substitution, which adds friction and breaks Docker Compose UX (docker compose ps, logs, etc. with stdin-based flows).

Describe the solution you'd like

Support the ref+<backend>://path#/key URI scheme in environment values, consistent with the convention already established by:

• helmfile/vals — the reference implementation of this URI scheme
• ArgoCD Vault Plugin
• helmfile

Example compose.yaml:

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: "ref+openbao://secret/data/prod/postgres#/username"
      POSTGRES_PASSWORD: "ref+openbao://secret/data/prod/postgres#/password"
      POSTGRES_DB: "ref+openbao://secret/data/prod/postgres#/database"

  myapp:
    image: myapp:latest
    environment:
      DATABASE_URL: "ref+vault://secret/data/prod/postgres#/connection_string"
      API_KEY: "ref+awssecrets://prod/api-keys#/stripe"
    depends_on:
      - postgres

The URI format follows the vals convention:

ref+<backend>://<path>#/<key>

Component	Description
ref+	Prefix indicating a secret reference (distinguishes from regular values)
<backend>	Secret store backend (openbao, vault, awssecrets, gcpsecrets, azurekeyvault, etc.)
<path>	Path to the secret in the backend
#/<key>	JSON pointer to extract a specific field from the secret

Correlation with helmfile/vals

The vals project defines this URI scheme and supports 20+ backends. Docker Compose could either:

1. Use vals as a library (github.com/helmfile/vals) — immediate support for all backends with minimal code
2. Implement the same URI scheme natively — starting with a few common backends (OpenBao, Vault, AWS Secrets Manager) using a pluggable resolver interface, keeping compatibility with the vals syntax so users have a consistent experience across tools

Either way, adopting the same ref+<backend>:// syntax means users who already use helmfile, ArgoCD Vault Plugin, or vals get a familiar, zero-learning-curve experience in Docker Compose.

Proposed implementation

Resolution would happen at project load time (after interpolation, before container creation), in pkg/compose/loader.go:

• Values prefixed with ref+ are detected and resolved against the appropriate backend
• Backend authentication uses standard environment variables (OPENBAO_ADDR/OPENBAO_TOKEN, VAULT_ADDR/VAULT_TOKEN, AWS_REGION, etc.)
• Non-ref+ values are untouched (fully backward compatible)
• A resolver interface makes adding new backends straightforward

Describe alternatives you've considered

• vals eval as preprocessor: Works but breaks normal Docker Compose workflows (can't use named projects, docker compose down, etc. properly with stdin)
• Shell variable substitution with wrapper scripts: Exposes secrets in process environment and shell history
• Docker secrets / external secret operators: Only works with Swarm or Kubernetes, not standalone Compose

Additional context

• The ref+ prefix makes this opt-in and backward compatible — existing compose files are unaffected
• OpenBao and HashiCorp Vault use nearly identical APIs (OpenBao is a fork), so supporting both is trivial
• This would make Docker Compose a first-class citizen in secret-store-backed workflows alongside Helm and ArgoCD

[johan-nilsson-82]

Note: this initial implementation uses github.com/helmfile/vals for broad backend support, with direct OpenBao/Vault handling to avoid the vals mount preflight behavior. I’m open to simplifying this if maintainers prefer either a pure vals-based approach, a smaller native resolver interface, or an explicit opt-in flag.