Dynamic volume provisioning folder per pod in a volume?

7/24/2020

What I am aiming for is a folder(pod name) per pod created inside of a volume using a volumeClaimsTemplate in a StatefulSet.

An example would be:

  • PersistentVolume = "/data"
  • Pods:
    • pod-0 = "/data/pod-0"
    • pod-1 = "/data/pod-1"

I am struggling with getting the replicas to create new folders for themselves. Any help with how to do this would be grateful.

-- StackDrenny
kubernetes
kubernetes-statefulset
minikube
persistent-volumes
provisioning

1 Answer

7/28/2020

volumeClaimTemplates is a list of claims that pods are allowed to reference. The StatefulSet controller is responsible for mapping network identities to claims in a way that maintains the identity of a pod. Every claim in this list must have at least one matching (by name) volumeMount in one container in the template. A claim in this list takes precedence over any volumes in the template, with the same name.

This means that with volumeClaimTemplates you can request the PVC from the storage class dynamically.

If we use this yaml as an example:

volumeClaimTemplates: 
  - metadata: 
      name: www
    spec: 
      accessModes: ["ReadWriteOnce"]
      storageClassName: "standard"
      resources: 
        requests: 
          storage: 1Gi
      

Once you deploy your pods you notice that your pods are being created and PVC is requested during the creation. PVC is name in the following convention:

volumeClaimTemplate name + Pod-name + Ordinal-number

So if you take above yaml as an example you will receive three PVC (assuming 3 replicas):

NAME		STATUS		VOLUME		
www-web-0  	Bound		pvc-12d77135...
www-web-1  	Bound		pvc-08724947...
www-web-2	Bound 		pvc-50ac9f96

It's worth mentioning that Persistent Volume Claims represent the exclusive usage of a Persistent Volume by a particular Pod. This means that if we look into the volumes individually we find that each is assign to a particular pod:

➜ ~ pwd
/tmp/hostpath-provisioner/pvc-08724947...
➜ ~ ls 
web-1
➜ ~ pwd
/tmp/hostpath-provisioner/pvc-50ac9f96...
➜ ~ ls 
web-2

While testing this I did achieve your goal but I had to create persistentvolumes manually and they had to point towards the same local path:

 local:
    path: /home/docker/data
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - minikube

This combined with subPathExpr mounted the directories named after the pods into the specified path.

    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html
      subPathExpr: $(NAME)
    env:
      - name: NAME
        valueFrom: 
          fieldRef:
             fieldPath: metadata.name

And the result of this (web was the name of the deployment):

➜ ~ pwd
/home/docker/data
➜ ~ pwd
web-0  web-1  web-2

Here`s more information how subpath with expanded env variables works.

Use the subPathExpr field to construct subPath directory names from Downward API environment variables. This feature requires the VolumeSubpathEnvExpansion feature gate to be enabled. It is enabled by default starting with Kubernetes 1.15. The subPath and subPathExpr properties are mutually exclusive.

Let me know if you have any questions.

-- acid_fuji
Source: StackOverflow