Kubernetes Secrets & GitOps

A secret store solves "where does the real value live," but a running pod still needs the value inside the cluster. This page covers Kubernetes' own Secret object, why it isn't encryption by itself, and how External Secrets, SOPS, and Sealed Secrets each close a different gap — especially the gap a GitOps workflow creates: infrastructure state lives in git, and a secret can't sit there in the clear.

TL;DR — in 30 seconds:

• A Kubernetes Secret's values are base64-encoded, not encrypted — decoding one is a one-line command; real protection is encryption at rest plus RBAC, both cluster-level settings you have to turn on deliberately.
• External Secrets Operator syncs a real secret from Vault/Secrets Manager/SSM into a Kubernetes Secret automatically — the source of truth stays in the external store, not in a manifest.
• SOPS and Sealed Secrets solve the GitOps problem by encrypting the value itself before it's committed — the encrypted blob is safe to put in git; only the right key (or the target cluster) can decrypt it back.

1. What a Kubernetes Secret actually is

flowchart LR
    A["kubectl create secret"] --> B["Secret object<br/>values base64-encoded"]
    B --> C["Mounted as env var<br/>or file in a pod"]
    B -.->|"encryption at rest?"| D["etcd<br/>(cluster's datastore)"]

A Secret is a Kubernetes API object, structurally almost identical to a ConfigMap, whose values are stored base64-encoded. Base64 is an encoding for safely representing binary data as text — it is not encryption; anyone who can kubectl get secret -o yaml a Secret (or read it directly from etcd, the cluster's backing datastore) can decode it in one command. The name "Secret" signals intent — "treat this as sensitive, apply RBAC to it" — it isn't a guarantee on its own.

Two settings turn that intent into real protection:

• Encryption at rest — a cluster-level setting so Secrets are actually encrypted where etcd stores them, not just base64-encoded.
• RBAC — restricting which users/service accounts can get/list Secret objects at all, so even an unencrypted-at-rest cluster limits who can read them through the API.

2. External Secrets Operator

Rather than a human (or a pipeline) writing a Secret manifest with the real value pasted in, the External Secrets Operator (ESO) runs inside the cluster, watches an ExternalSecret resource that references a path in Vault/AWS Secrets Manager/SSM Parameter Store, and syncs the real value into a native Kubernetes Secret automatically, keeping it up to date if the source rotates. The ExternalSecret resource itself — which only contains a reference (a path), never the value — is safe to commit to git; the Secret it generates is not, and stays out of version control entirely.

flowchart LR
    A["ExternalSecret resource<br/>(reference only — safe in git)"] --> B["External Secrets Operator<br/>watches the reference"]
    B --> C["Vault / Secrets Manager / SSM<br/>(real value lives here)"]
    C --> D["Native Kubernetes Secret<br/>synced automatically"]
    D --> E["Pod<br/>mounts as env var or file"]
    C -.->|"value rotates"| D

3. SOPS

SOPS (Secrets OPerationS) encrypts specific values inside a YAML/JSON/ENV file — keys stay readable, values are replaced with ciphertext — using a key from a key-management backend (AWS KMS, GCP KMS, PGP, age). The encrypted file is committed to git as-is; decrypting it back to a usable manifest requires access to that key, which only authorized people/systems have. A diff of a SOPS-encrypted file in git still shows which keys changed, just not the values — useful for review without exposing anything.

4. Sealed Secrets

Sealed Secrets (from Bitnami) takes a different approach purpose-built for Kubernetes: a client-side kubeseal tool encrypts a plain Secret manifest into a SealedSecret custom resource, using the target cluster's public key. The SealedSecret is safe to commit to git — only the controller running inside that specific cluster, which holds the matching private key, can decrypt it back into a real Secret. This ties the encrypted secret to exactly one cluster by design, which is either exactly what you want (no key management burden beyond the cluster itself) or a limitation (re-sealing needed per cluster), depending on the setup.

The two approaches both keep the encrypted form safe in git, but differ in where the decryption key lives — that's the real choice between them:

flowchart LR
  subgraph sops["SOPS"]
    s1["Plain value"] --> s2["Encrypted with a KMS/PGP/age key"] --> s3["Committed to git"]
    s3 --> s4["Decrypted by anyone/anything holding that key"]
  end
  subgraph sealed["Sealed Secrets"]
    k1["Plain Secret manifest"] --> k2["kubeseal encrypts with the cluster's public key"] --> k3["SealedSecret committed to git"]
    k3 --> k4["Decrypted only by that cluster's controller (private key)"]
  end

5. Choosing an approach

Need	Reach for
Source of truth stays in an external store, cluster stays in sync	External Secrets Operator
Encrypt values across many file types (not just K8s), flexible key backends	SOPS
Kubernetes-native, no external key-management system to run	Sealed Secrets

These aren't mutually exclusive — a real setup often uses ESO to pull from a central store for most secrets, and SOPS or Sealed Secrets for the handful that genuinely need to be defined directly in the GitOps repo.

Common gotchas

• Committing a plain Secret manifest to a GitOps repo "temporarily." Fix: there's no safe version of this — encrypt with SOPS or Sealed Secrets first, or use External Secrets so the real value never needs to be in a manifest at all.
• Assuming base64 in a Secret is "basically encrypted." Fix: base64 is reversible with a single base64 -d — treat a Secret's real protection as coming from encryption at rest and RBAC, both of which have to be explicitly configured.
• Losing the Sealed Secrets controller's private key. Fix: without it, every SealedSecret ever created for that cluster becomes permanently undecryptable — back up the controller's key material the same way you'd back up any other critical credential.

Check yourself — a SealedSecret and a plain Secret manifest both end up in the same public git repo. Which one is actually safe to be there, and why?

Only the SealedSecret — its values are encrypted with the target cluster's public key, and only the controller holding the matching private key inside that cluster can decrypt it. The plain Secret's values are merely base64-encoded, which anyone can reverse instantly.

Check yourself — what's the practical difference between using External Secrets Operator versus SOPS for a given secret?

External Secrets Operator keeps the real value out of git entirely — the store (Vault/Secrets Manager/SSM) stays the single source of truth, and only a reference to it lives in the repo. SOPS instead lets the encrypted value itself live in git, decrypted only when needed via a key — useful when you want the secret's definition to live in the GitOps repo itself, not in a separate external system.

You can defend this when you can explain exactly what a Kubernetes Secret protects (and doesn't) by default, and can pick between External Secrets, SOPS, and Sealed Secrets for a given GitOps scenario by naming the specific gap each one closes.