Apparenlty this behavior is caused by ServiceAccount
: https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller
Which uses something called an AdmissionController
. I guess what I'm looking for is one of the following:
Find a setting in AdmissionController
which skips the secrets mount for a given container (initContainer) in my case
Find an implementation of AdmissionController
which has this flexibility
Change the location of secrets from /var/run/secrets to somewhere else
I have an initContainer which is a part of a pod a part of statefulset. I am mounting some straight forward volumes (so I can create paths/permissions before my app container starts). However as soon as I check the file system, I see a nested path with what seems to be kubernetes secrets.
How did this get mounted? Is this our own doing? Why this path? Can I stop the secrets from being mounted? Can i change the mount path?
$ kubectl logs nmnode-0-0 -n test -c prep-hadoop-paths
drwxrwsrwt 4 root root 80 Aug 21 03:52 /run
/run:
total 0
drwxrwsr-x 2 1000 root 40 Aug 21 03:52 configmaps
drwxr-sr-x 3 root root 60 Aug 21 03:52 secrets
/run/configmaps:
total 0
/run/secrets:
total 0
drwxr-sr-x 3 root root 60 Aug 21 03:52 kubernetes.io
/run/secrets/kubernetes.io:
total 0
drwxrwsrwt 3 root root 140 Aug 21 03:51 serviceaccount
/run/secrets/kubernetes.io/serviceaccount:
total 0
lrwxrwxrwx 1 root root 13 Aug 21 03:51 ca.crt -> ..data/ca.crt
lrwxrwxrwx 1 root root 16 Aug 21 03:51 namespace -> ..data/namespace
lrwxrwxrwx 1 root root 12 Aug 21 03:51 token -> ..data/token
initContainers:
- command:
- sh
- -c
- umask 002; ls -ld /run; ls -lR /run; mkdir -p /var/run/secrets/credentials
; mkdir -p /var/opt/hdfs ; mkdir -p /var/run/configmaps ; mkdir -p /var/run/secrets/certificates
; ls -lR /var;
image: ubuntu:16.04
imagePullPolicy: IfNotPresent
name: prep-hadoop-paths
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/opt
name: data
subPath: hadoop/var/opt
- mountPath: /var/log
name: logs
subPath: hadoop
- mountPath: /var/run
name: var-run
subPath: hadoop
As you can see from the initContainer spec, there is nowhere that I specify or require any secrets to be mounted. However they show up regardless
The following is the volumes listing for the pod.
volumes:
- name: mssql-master-pool-secret
secret:
defaultMode: 420
secretName: mssql-master-pool-secret
- name: controller-internal-secret
secret:
defaultMode: 420
secretName: controller-internal-secret
- emptyDir:
medium: Memory
name: var-run
- configMap:
defaultMode: 420
name: mssql-hadoop-storage-0-configmap
name: hadoop-config-volume
- name: nmnode-0-agent-secret
secret:
defaultMode: 420
secretName: nmnode-0-agent-secret
- configMap:
defaultMode: 420
name: cluster-configmap
name: cluster-config-volume
If you need more parts of the yaml please let me know.
I believe that every pod runs under the context of a service account. That secret that got mounted is the service account's token. Is there a reason why you don't want that to happen?
You are on a right way, as Admission controller is the main contributor for implementing various features via Kubernetes API. As you mentioned above Service Account is also propagated with some admission features in Kubernetes cluster, initially bounded to the particular admission plugin called ServiceAccount
.
According to the official Kubernetes documentation, there are special flags--enable-admission-plugins
and --disable-admission-plugins
included in kube-apiserver
configuration that can be used to enable or disable admission plugins respectively.
By default, ServiceAccount
admission controller plugin is enabled like described here. Besides other actions, this plugin mounts volume with token data and CA certificate into the each Pod across K8s cluster for authentication to the apiserver purposes.
In order to deactivate ServiceAccount
admission plugin, you can inject --disable-admission-plugins=ServiceAccount
into the kube-apiserver
configuration.
Otherwise, if you want to
Change the location of secrets from /var/run/secrets to somewhere else
This is where you can change the path to whatever you want