How are kubernetes secrets mounted?

8/21/2019

Update

Apparenlty this behavior is caused by ServiceAccount: https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller

Which uses something called an AdmissionController. I guess what I'm looking for is one of the following:

  • Find a setting in AdmissionController which skips the secrets mount for a given container (initContainer) in my case

  • Find an implementation of AdmissionController which has this flexibility

  • Change the location of secrets from /var/run/secrets to somewhere else

Original

I have an initContainer which is a part of a pod a part of statefulset. I am mounting some straight forward volumes (so I can create paths/permissions before my app container starts). However as soon as I check the file system, I see a nested path with what seems to be kubernetes secrets.

How did this get mounted? Is this our own doing? Why this path? Can I stop the secrets from being mounted? Can i change the mount path?

$ kubectl logs nmnode-0-0 -n test -c prep-hadoop-paths
drwxrwsrwt 4 root root 80 Aug 21 03:52 /run
/run:
total 0
drwxrwsr-x 2 1000 root 40 Aug 21 03:52 configmaps
drwxr-sr-x 3 root root 60 Aug 21 03:52 secrets

/run/configmaps:
total 0

/run/secrets:
total 0
drwxr-sr-x 3 root root 60 Aug 21 03:52 kubernetes.io

/run/secrets/kubernetes.io:
total 0
drwxrwsrwt 3 root root 140 Aug 21 03:51 serviceaccount

/run/secrets/kubernetes.io/serviceaccount:
total 0
lrwxrwxrwx 1 root root 13 Aug 21 03:51 ca.crt -> ..data/ca.crt
lrwxrwxrwx 1 root root 16 Aug 21 03:51 namespace -> ..data/namespace
lrwxrwxrwx 1 root root 12 Aug 21 03:51 token -> ..data/token
      initContainers:
      - command:
        - sh
        - -c
        - umask 002; ls -ld /run; ls -lR /run; mkdir -p /var/run/secrets/credentials
          ; mkdir -p /var/opt/hdfs ; mkdir -p /var/run/configmaps ; mkdir -p /var/run/secrets/certificates
          ; ls -lR /var;
        image: ubuntu:16.04
        imagePullPolicy: IfNotPresent
        name: prep-hadoop-paths
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/opt
          name: data
          subPath: hadoop/var/opt
        - mountPath: /var/log
          name: logs
          subPath: hadoop
        - mountPath: /var/run
          name: var-run
          subPath: hadoop

As you can see from the initContainer spec, there is nowhere that I specify or require any secrets to be mounted. However they show up regardless

The following is the volumes listing for the pod.

      volumes:
      - name: mssql-master-pool-secret
        secret:
          defaultMode: 420
          secretName: mssql-master-pool-secret
      - name: controller-internal-secret
        secret:
          defaultMode: 420
          secretName: controller-internal-secret
      - emptyDir:
          medium: Memory
        name: var-run
      - configMap:
          defaultMode: 420
          name: mssql-hadoop-storage-0-configmap
        name: hadoop-config-volume
      - name: nmnode-0-agent-secret
        secret:
          defaultMode: 420
          secretName: nmnode-0-agent-secret
      - configMap:
          defaultMode: 420
          name: cluster-configmap
        name: cluster-config-volume

If you need more parts of the yaml please let me know.

-- Jeff Saremi
kubernetes

2 Answers

8/21/2019

I believe that every pod runs under the context of a service account. That secret that got mounted is the service account's token. Is there a reason why you don't want that to happen?

-- Jamie
Source: StackOverflow

8/22/2019

You are on a right way, as Admission controller is the main contributor for implementing various features via Kubernetes API. As you mentioned above Service Account is also propagated with some admission features in Kubernetes cluster, initially bounded to the particular admission plugin called ServiceAccount.

According to the official Kubernetes documentation, there are special flags--enable-admission-plugins and --disable-admission-plugins included in kube-apiserver configuration that can be used to enable or disable admission plugins respectively.

By default, ServiceAccount admission controller plugin is enabled like described here. Besides other actions, this plugin mounts volume with token data and CA certificate into the each Pod across K8s cluster for authentication to the apiserver purposes.

In order to deactivate ServiceAccount admission plugin, you can inject --disable-admission-plugins=ServiceAccount into the kube-apiserver configuration.

Otherwise, if you want to

Change the location of secrets from /var/run/secrets to somewhere else

This is where you can change the path to whatever you want

-- mk_sta
Source: StackOverflow