Kubernetes - Where does it store secrets and how does it use those secrets on multiple nodes?

1/19/2018

Not really a programming question but quite curious to know how does Kubernetes or Minikube manage secrets & uses it on multiple nodes/pods?

Let's say if I create a secret to pull image with kubectl as below -

$ kubectl create secret docker-registry regsecret --docker-server=https://index.docker.io/v1/ --docker-username=$USERNM --docker-password=$PASSWD --docker-email=vivekyad4v@gmail.com

What processes will occur in the backend and how will k8s or Minikube use those on multiple nodes/pods?

-- vivekyad4v
kubernetes
minikube

3 Answers

1/19/2018

Secrets are stored in the only datastore a kubernetes cluster has: etcd.

As all other resources, they're retrieved when needed by the kubelet executable (that runs in every node) by querying k8s' API server

-- whites11
Source: StackOverflow

7/8/2019

If you are wandering how to actually access the secrets (the stored files),

kubectl -n kube-system exec -it <etcd-pod-name> ls -l /etc/kubernetes/pki/etcd

You will get a list of all keys (system default keys). you can simply view them using cat command (if they are encrypted you won't see much)

-- David Wer
Source: StackOverflow

1/19/2018

All data in Kubernetes is managed by the API Server component that performs CRUD operations on the data store (current only option is etcd).

When you submit a secret with kubectl to the API Server it stores the resource and data in etcd. It is recommended to enable encryption for secrets in in the API Server (through setting the right flags) so that the data is encrypted at rest, otherwise anyone with access to etcd will be able to read your secrets in plain text.

When the secret is needed for either mounting in a Pod or in your example for pulling a Docker image from a private registry, it is requested from the API Server by the node-local kubelet and kept in tmpfs so it never touches any hard disk unencrypted.

Here another security recommendation comes into play, which is called Node Authorization (again set up by setting the right flags and distributing certificates to API Server and Kubelets). With Node Authorization enabled you can make sure that a kubelet can only request resources (incl. secrets) that are meant to be run on that specific node, so a hacked node just exposes the resources on that single node and not everything.

-- puja
Source: StackOverflow