Securing Secrets in Kubernetes

Option 1 — Encrypt your secret text and check-in the encrypted secret into the Git

  1. Encrypt your plain text sensitive data using symmetric or asymmetric algorithm.
  2. Use k8s Custom Resource Definition (CRD) to create custom secret resource using the encrypted secret content.
  3. Create a custom k8s controller which reads the encrypted secret object and at runtime decrypts it and creates a k8s native secret object.

Option 2 — Use external secret management service for storing the secrets

  1. You can store the sensitive data to 3rd part secret management service like AWS Secrets Manager or HashiCorp Vault.
  2. Create custom k8s controllers which can fetch secrets from these services based on some configuration and create k8s native secret object at runtime.

Quick overview of Sealed Secrets —

With Sealed Secret open source project — you can encrypt your Secret into a SealedSecret, which is safe to store — even to a public repository. The SealedSecret can be decrypted only by the controller running in the target cluster and nobody else (not even the original author) is able to obtain the original Secret from the SealedSecret.

  • A cluster-side controller / operator
  • A client-side utility: kubeseal

Step 1

Install kubeseal, a client utility tool which helps you to create SealedSecret custom Kubernetes resource from the Secret resource manifest file.

> wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.15.0/kubeseal-linux-amd64 -O kubeseal> sudo install -m 755 kubeseal /usr/local/bin/kubeseal

Step 2

Install Custom Controller and Custom Resource Definition (CRD) for SealedSecrets

> kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.15.0/controller.yaml
rolebinding.rbac.authorization.k8s.io/sealed-secrets-service-proxier created
role.rbac.authorization.k8s.io/sealed-secrets-key-admin created
clusterrole.rbac.authorization.k8s.io/secrets-unsealer created
deployment.apps/sealed-secrets-controller created
customresourcedefinition.apiextensions.k8s.io/sealedsecrets.bitnami.com created
service/sealed-secrets-controller created
clusterrolebinding.rbac.authorization.k8s.io/sealed-secrets-controller created
serviceaccount/sealed-secrets-controller created
role.rbac.authorization.k8s.io/sealed-secrets-service-proxier created
rolebinding.rbac.authorization.k8s.io/sealed-secrets-controller created

Step 3

Verify if the sealed-secret controller pod is running

> kubectl get pods -n kube-system -l name=sealed-secrets-controllerNAME                                         READY   STATUS RESTARTS   AGE
sealed-secrets-controller-7c766b885b-d5r2r 1/1 Running 0 7m39s
> kubectl logs sealed-secrets-controller-7c766b885b-d5r2r -n kube-systemcontroller version: v0.15.0
2021/05/01 20:13:34 Starting sealed-secrets controller version: v0.15.0
2021/05/01 20:13:34 Searching for existing private keys
2021/05/01 20:13:35 New key written to kube-system/sealed-secrets-keymt6dg
2021/05/01 20:13:35 Certificate is
-----BEGIN CERTIFICATE-----
MIIErjCCApagAwIBAgIRAJqYfaZsali26I8pvBXoFGYwDQYJKoZIhvcNAQELBQAw
ADAeFw0yMTA1MDEyMDEzMzVaFw0zMTA0MjkyMDEzMzVaMAAwggIiMA0GCSqGSIb3
DQEBAQUAA4ICDwAwggIKAoICAQDp/yO5PY8ACHBDuguhtfpOwlbScK9hZorJloyx
ixVCc57j1zMSX0pSVcrk1Yuyf6sYvBQtDi16kM70z6y/ODiz+9g87K/jY7B0UAoi
mpzM/T0tWJiG9ixyNMZhHoNREauokSlbERq3Jl8ZNTfmxHWhLH7DhkJ7MdpQfMpK
a3XHcSZyz1mXFqv+OSCCwllWCRHmHgp/vqudAv8+NYm0gnAxKt2fjlv/ObX8J1RI
CtLnlsCpp/9SyVcSTeYYaqjUsI7fTUZ7tkTE/bdQHwf3xe4DhUty7xLqMF1OPSPw
EetL8fGO0VqoSQFKQ0Bf78+8vhAA2cwkuqB6vQQm9pT3yC5niSCUo+jwFcfyknjr
yx8DINbq6K9B40EXh8X7w4I6zwYpyT0GoNU54wW0ki8pHRm7EnFeBOkUvNspzmKn
t/EZEDVq74Kkl/BRNRvKHYlwudSoJuvvX6JM8DVvRp0lMPnXnG3RLSmCP3gEFQBZ
DhbnkwO+6ADX9Q4vyqelWoHWdVGVULDlMDhSzvEhFFgPcZXzWTShH81vfl8M6lpT
U0ysZkA6i3A29XEJpPj35yWPBDWmKF5fLM3ChMt/NSJEeoJN1RboPDAgVUTxEW59
q+Tq09/zlYD7Ch8PNc3IWNXjFNXmCAAOw9Z1VBbD8p6LrC5JvBtPoWYqufWVXQD9
KDe+6wIDAQABoyMwITAOBgNVHQ8BAf8EBAMCAAEwDwYDVR0TAQH/BAUwAwEB/zAN
BgkqhkiG9w0BAQsFAAOCAgEAvu//VzDREYZPPIW1maTxo9C/nHEEuOP0rQU3zVQr
bBYf8N0b5wpCllESCgi0JDJJXrE8KrjfdtawjoBrBlHOdWHF+fIot2KbrC/i37em
/ulMAgiiJzrKM/ExJuCuH60fsSIx4wrg46tQpU8jHFWq7nGnsaE+UN3QPjuvQ+qo
KKDSBLDxLx+q9vBfaXElblh4okUI8Pr4UEEJrYiPzPM6nA9EPpy53N3si4jyDJJb
2IsCUa2bW6iBhpyZOQQUPn22ziWRQ/sYYNmtP/gX0rwtk+Rr8TTdzPYGZcYfMQ6O
TFq4Zo2/TnpCL/CUr2DiSuF2qdWGGvbQOENYq2FNuDI4zeljElcZHXA8nhpbNSJs
7VNqqz5ZTFCKyL0Gn6SawGT7EdwBT2AD3F33Qd/7bXG/On7KdVw6FKHbZOR2RcoS
YFQv7Xr8g/4atQjxDa7R5+zkxd5unsvpFhYM1UfNJc4cjJ7SmfCCHoPGiwZ0OgqB
6SvUVU64QmMMJ/jYAJkYMOakSHaRITHAvvBjpAMKxSjjb7qZD5FnpXLhRY9lNiY6
MnnQRxJskCw+R6geIAHTMzAofMfc1haIEr+3oMFZfyh1LFFsz3B4hMxXYKrWYDje
+96bhAY9X7L0UfREjmw8HCeZneEuBJjX9z/PyIeMdhViLh9uO/MAL1MBxdBVA55+
LW8=
-----END CERTIFICATE-----
2021/05/01 20:13:35 HTTP server serving on :8080
> kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml

Step 4

Create a Secret manifest file with name secrets.yaml. We want to secure the DB_USER and DB_PWD data.

apiVersion: v1
data:
DB_PWD: cGFzc3dvcmQ= //base64 encoded
DB_USER: cm9vdA== //base64 encoded
kind: Secret
metadata:
name: db-secrets
> kubeseal --format=yaml < secret.yaml > sealed-secret.yaml> cat sealed-secret.yaml apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: db-secrets
namespace: default
spec:
encryptedData:
DB_PWD: AgDaCRi27RV4/sVI2ok7JlqBSKT5+c7gGJog+...
DB_USER: AgAZG67CrrOBnyKIKha7xhJulr+CQGPaE/PpsjvY8jJR0IDO2...
template:
metadata:
creationTimestamp: null
name: db-secrets
namespace: default

Step 5

Let’s create resource in k8s using the Sealed Secret manifest file.

> kubectl apply -f sealed-secret.yaml
> kubectl logs sealed-secrets-controller-7c766b885b-d5r2r -n kube-system2021/05/01 20:38:06 Updating default/db-secrets
2021/05/01 20:38:06 Event(v1.ObjectReference{Kind:"SealedSecret", Namespace:"default", Name:"db-secrets", UID:"fd89a7e7-c81a-4110-9de6-6b65195169d3", APIVersion:"bitnami.com/v1alpha1", ResourceVersion:"19365", FieldPath:""}): type: 'Normal' reason: 'Unsealed' SealedSecret unsealed successfully

--

--

#AWS #CloudArchitect #CloudMigration #Microservices #Mobility #IoT

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Naresh Waswani

Naresh Waswani

#AWS #CloudArchitect #CloudMigration #Microservices #Mobility #IoT