feat: resolve ref+ secret references in environment values

[johan-nilsson-82]

What I did

Add support for resolving ref+openbao:// secret references in service environment values.

The implementation resolves references at project load time using the OpenBao API client.

Currently this supports OpenBao KV v2-style secret paths.

Related issue

Closes #13821

[glours]

Hi @johan-nilsson-82
Thanks for this PR. We've been looking at adding external secret store support to Compose and your contribution is actually good evidence that this is a real need — we're citing it internally.

The direction we're taking is native support for Docker Secret Engine (se://), a backend-agnostic daemon that proxies to multiple vaults (1Password today, Vault/OpenBao/Bitwarden on the roadmap). That way Compose only ever needs one integration, rather than a separate SDK dependency for each backend.

Given that, we're unlikely to merge this approach, but this isn't a no to the goal, just a different path to it. The design is in progress and your input would be welcome.

[johan-nilsson-82]

Thanks, that direction makes sense.

One clarification from my side: in #13821 I mentioned vals as one possible implementation option, but I intentionally did not use it in this PR.

The main reasons were:

• I did not want to force a Go/toolchain version bump in Compose.
• Pulling in vals directly also brings a large dependency graph, because it supports many backends.
• That seemed too heavy for Compose itself, especially for an initial proof of concept.

So the PR took the smaller native route: implement only the minimal OpenBao/Vault-style resolver path directly, while keeping the user-facing syntax compatible with the vals / Helmfile convention.

Given the Docker Secret Engine direction, I agree that the backend integrations probably belong outside Compose. In that model, vals may still be useful as prior art, and possibly even as a good starting point for early compatibility work, but probably not as a direct Compose dependency.

One thing I think is worth considering in the se:// design is how backend-specific paths and mixed backends would be represented.

For example, with the vals-style syntax the backend and path semantics are explicit:

environment:
  POSTGRES_PASSWORD: "ref+openbao://secret/data/prod/postgres#/password"
  API_KEY: "ref+awssecrets://prod/api-keys#/stripe"

Different secret stores do not always have the same path model, field selector model, or authentication context. Some projects may also need to mix secret stores, for example OpenBao for internal app secrets and AWS Secrets Manager for cloud-provider-managed secrets.

A generic se://... reference may be cleaner from the Compose side, but it also seems like it would hide those backend details behind separate Docker Secret Engine configuration. That may be the right abstraction, but it would be useful to understand how users will:

• choose which backend a given se:// reference resolves against
• represent backend-specific path conventions
• mix multiple vaults/backends in the same Compose project
• migrate from existing ref+<backend>://... references

So I think the important question is not whether Compose should depend on vals directly, but whether Docker Secret Engine could learn from the vals URI model or provide a compatibility/import/migration path for users already using Helmfile/vals-style references.

[glours]

@johan-nilsson-82 thanks for your feedback, I'll close the PR and keep the issue open