Kubernetes Secrets Management Failures
The Secret was locked down. Its value was in a crash log.
What Is Kubernetes Secrets Management Failures?
A Kubernetes Secret is base64 rather than encryption, and a Secret injected as an environment variable travels with the process environment into logs and crash dumps. You'll be refused the Secret itself, find its value printed in a crash log you were allowed to read, and use it against a payout API. Then you'll mount the key as a file, encrypt Secrets at rest with a KMS provider and rotate the leaked key. Maps to OWASP Kubernetes Top Ten K03, Secrets Management Failures.
What You'll Learn in Kubernetes Secrets Management Failures
- Explain that base64 encoding offers no confidentiality.
- Recognize how Secrets passed as environment variables leak into logs and crash dumps.
- Mount Secrets as read-only files and enable encryption at rest with a KMS provider.
- Rotate a leaked credential, since configuration changes do not recall an existing copy.
Kubernetes Secrets Management Failures — Training Steps
-
Try the Secret directly
Bob holds a Morrowkin developer's kubeconfig, taken from a compromised laptop. Developers can read pods and logs in the payments namespace. He tries the obvious first: reading the payout service's Secret. Morrowkin moves money from card payments into merchants' bank accounts.
-
A worker that keeps crashing
Developers may read pods. One of the payout workers has restarted many times.
-
Read the crash log
Developers may read logs too. Bob reads the output of the crashed worker's last run.
-
Use the payout key
Bob calls the payout API with the key, from his own machine.
-
Check your understanding
-
A payout destination changed
Morrowkin's fraud team reports that a merchant's payout destination was changed through the API by an unknown caller using the production key.
-
Open the payout worker
Alice opens the payout worker's deployment and the API server's encryption configuration.
-
A Secret turned into an environment variable
The key is read from the Secret, but it is handed to the process as an environment variable. Environment variables are inherited by child processes, shown by debugging tools and, here, printed by a crash handler.
-
Mount the key as a file
Alice mounts the Secret as a read-only file and tells the worker where to read it. A file is not part of the environment, so a dump of the environment no longer contains the key.
-
Encryption at rest is off
Secrets are stored in etcd. With the identity provider first, they are stored there exactly as they arrive: base64, readable by anyone who can read etcd or its backups.
Security Framework Coverage
OWASP Kubernetes Top 10
- K03:2025 Secrets Management Failures
CWE
- CWE-312 Cleartext Storage of Sensitive Information
- CWE-522 Insufficiently Protected Credentials
MITRE ATT&CK
- T1552.007 Unsecured Credentials: Container API
CIS Controls
- CIS 3 Data Protection
- CIS 4 Secure Configuration of Enterprise Assets and Software
NIST CSF
- PR.AT-02 Individuals in specialized roles are provided with awareness and training so that they possess the knowledge and skills to perform relevant tasks with cybersecurity risks in mind
- PR.DS Data Security
- PR.PS Platform Security
NICE Framework
- IO-WRL-005 Systems Administration
- DD-WRL-001 Cybersecurity Architecture