Secrets with SOPS

You add secrets: - secrets/prod.yaml to a release, run helmfile diff in CI, and the database password that file was supposed to protect scrolls straight down the build log in plaintext. Encrypting a file at rest never promised anything about what happens after it's decrypted — and that gap is exactly where this page slows down: what SOPS actually encrypts, how helm-secrets bridges it into Helmfile, what a .sops.yaml rule decides, and the failure modes that show up once a team (not just a laptop) needs to decrypt.

TL;DR — in 30 seconds:

• SOPS, helm-secrets, and Helmfile are three separate tools chained together — SOPS encrypts leaf values, helm-secrets decrypts them via a key backend, Helmfile just orchestrates the hand-off.
• SOPS only encrypts leaf values, not the YAML's keys or shape — a git diff on an encrypted file still shows which key changed.
• helm-secrets decrypts into memory only, but helmfile diff/--debug output and shell history can still leak plaintext — pass --suppress-secrets in any CI log that isn't private.

1. Three tools, one job

"Secrets in Helmfile" is really three separate tools chained together, each with a narrow responsibility:

Tool	Responsibility	Knows about Helmfile?
SOPS	Encrypts/decrypts individual leaf values in a YAML/JSON/INI file, using a key from a backend (age, KMS, PGP)	No — a general-purpose file encryptor
helm-secrets	A Helm plugin that finds secrets: entries, shells out to SOPS to decrypt them, and feeds the plaintext to Helm	No — a Helm plugin, reusable outside Helmfile
Helmfile	Reads the secrets: key on a release or environment, hands each entry to helm-secrets before the Helm call	Orchestrates the other two

None of the three requires the others to exist independently — you could run sops by hand, or use helm-secrets with plain helm and no Helmfile. Helmfile's contribution is just: treat secrets: as a second values list, decrypted immediately before use.

flowchart LR
    SecretsKey["release/environment<br/>secrets: list"]
    HelmSecrets["helm-secrets plugin"]
    Sops["sops CLI"]
    Backend["key backend<br/>(age / KMS / PGP)"]
    Plaintext["plaintext values<br/>(in memory only)"]
    Helm["helm upgrade --install"]

    SecretsKey --> HelmSecrets --> Sops --> Backend
    Backend --> Plaintext --> Helm

2. What actually gets encrypted

SOPS encrypts a file's leaf values, not the file itself — keys, structure, and YAML shape stay plaintext:

# secrets/prod.yaml, before sops --encrypt --in-place
db:
  password: hunter2
  url: postgres://prod.example/app

# after sops --encrypt --in-place — structure is unchanged, values are ciphertext
db:
  password: ENC[AES256_GCM,data:Ay8k...,iv:...,tag:...,type:str]
  url: ENC[AES256_GCM,data:Qw1z...,iv:...,tag:...,type:str]

This is deliberate: a git diff on an encrypted file still shows which key changed, even though it can't show the new value — a meaningful signal in code review without exposing the secret itself. SOPS also appends a sops: metadata block (the encrypted data key, the backend used, a MAC over the file) that you should never hand-edit.

3. .sops.yaml decides who encrypts with what

A .sops.yaml at the repo root maps file paths to key backends via creation_rules:, so sops --encrypt doesn't need --age or --kms flags on every call:

# .sops.yaml
creation_rules:
  - path_regex: secrets/prod.*\.yaml$
    kms: arn:aws:kms:eu-west-1:123456789012:key/abc
  - path_regex: secrets/staging.*\.yaml$
    kms: arn:aws:kms:eu-west-1:123456789012:key/def
  - path_regex: secrets/.*\.yaml$
    age: age1qz...                             # fallback for anything else under secrets/

Rules are matched top to bottom, first match wins — order the more specific path_regex entries before the general fallback, the same way you'd order a switch statement. Decryption doesn't need .sops.yaml at all: the file's own sops: metadata block already records which backend and key encrypted it, so sops decrypt (or helm-secrets calling it for you) can always find its way back — .sops.yaml only matters for the next encryption, not for reading an existing file.

4. Two key backends, two operational models

Backend	Setup	Who can decrypt	Fits
age	One keypair per person (age-keygen), public key goes in .sops.yaml	Anyone whose public key is listed as a recipient	Laptop-only, small teams, no cloud dependency
KMS (AWS/GCP/Azure)	A cloud key resource; .sops.yaml references its ARN/ID	Anyone whose IAM/cloud identity has decrypt permission on that key	Teams and CI, decrypt authorization lives in the cloud, not on a laptop

The practical difference: with age, losing a laptop means losing a private key, and rotating access means re-encrypting every file for the new recipient list. With KMS, access is an IAM policy — revoke a role, decryption stops, no file needs to change. Most teams start on age and move to KMS once more than a handful of people (or any CI runner) need to decrypt.

flowchart TB
    subgraph Age["age backend"]
        AgeKey["private key file<br/>on each person's laptop"]
    end
    subgraph Kms["KMS backend"]
        IamRole["IAM role / cloud identity"]
        KmsKey["cloud KMS key"]
        IamRole -->|"decrypt permission"| KmsKey
    end
    File["encrypted values file"]
    AgeKey --> File
    KmsKey --> File

5. Referencing encrypted files from helmfile.yaml

secrets: accepts the same shapes as values: — a list of files, resolved relative to the helmfile.yaml that declares it — on a release, on an environment, or both:

releases:
  - name: demo-app
    chart: ./charts/demo-app
    values:
      - releases/demo-app.yaml
    secrets:
      - secrets/prod.yaml            # release-scoped secret

environments:
  prod:
    values:
      - environments/prod.yaml
    secrets:
      - secrets/prod-env.yaml        # environment-scoped secret, becomes .Environment.Values

The two land in different places: a release's secrets: merges into that release's Helm values (same tree as its values: list); an environment's secrets: merges into .Environment.Values, exactly like environment values: does — see the "Environments & values layering" page for how that tree gets consumed by templated values files.

6. Never touches disk in plaintext — but logs and shells still can

helm-secrets decrypts into memory and pipes straight to Helm; the plaintext file never lands on disk. That guarantee doesn't automatically extend to everywhere the value travels afterwards:

• Diff output. helmfile diff prints the rendered values, which would include decrypted secrets unless you pass --suppress-secrets — non-negotiable in any CI log that isn't private.
• --debug. Printing the underlying helm invocations can include --set values if a secret ever ends up there — prefer values:/secrets: files over set: for anything sensitive.
• Shell history and process lists. Passing a secret via an environment variable or CLI flag (instead of through the secrets: file mechanism) risks it showing up in ps output or a shell's history file.

The rule of thumb: if a value is sensitive, it belongs in a secrets:-referenced file, decrypted only by helm-secrets, never typed on a command line or exported as a bare env var.

7. How this connects

• §5 of "Environments & values layering" only covers where secrets: sits in the merge order — how the ciphertext it points to actually gets decrypted is this page's whole job; read that page first if the values:/secrets: distinction itself is still unclear.
• The release DAG page, next in this course, is where needs: ordering and --suppress-secrets meet: a release that decrypts a secret still has to apply in the right order relative to the releases that depend on it.
• Ansible's Vault (module 08, Ansible) solves the same "secret at rest in git" problem with a different trade-off: it encrypts a whole file as one opaque blob — no partial diff, which is why that module needs the vault-indirection naming trick just to keep references greppable — where SOPS's per-leaf-value encryption keeps keys visible in a git diff for free.

Common gotchas

• Assuming .sops.yaml controls decryption. Fix: it only decides which backend/key a new encryption uses — decrypting an existing file relies on the sops: metadata block already inside it, not on .sops.yaml.
• Running helmfile diff in CI without --suppress-secrets. Fix: diff prints rendered values, including any decrypted secret, straight into the build log unless you suppress them.
• Passing a secret via --set or a bare environment variable instead of a secrets: file. Fix: those paths risk showing up in --debug output, ps, or shell history — keep sensitive values only inside a secrets:-referenced file.
• Picking age for a team/CI setup. Fix: age keys live per-laptop, so rotating access means re-encrypting every file for the new recipient list — KMS ties decrypt access to an IAM policy instead, which is what scales past a handful of people or any CI runner.

Check yourself — a teammate says encrypting secrets/prod.yaml with SOPS makes the whole file unreadable in a diff. True?

No — SOPS only encrypts the leaf values; keys and YAML structure stay plaintext, so a git diff still shows exactly which key changed, just not the new value.

Check yourself — your team just grew from 3 laptops to a CI runner that also needs to decrypt secrets. Age or KMS?

KMS — decrypt access becomes an IAM policy on a cloud identity (including the CI runner's role) rather than a private key file that has to be distributed to and rotated on every machine that needs access.

You can defend this when you can explain what SOPS actually encrypts (leaf values, not the file shape), why .sops.yaml only matters for new encryptions and not for reading existing files, when you'd pick age over KMS for a team's key backend, and why --suppress-secrets is required in any CI pipeline whose logs aren't private.

Go deeper: the full creation_rules: syntax (including encrypted_regex and per-path key groups) lives in the SOPS project docs, and the plugin's own flags (--suppress-secrets, backend auto-detection) are in the helm-secrets docs — this page covers the 80/20 you need for the Do exercise.