K
Q

Template PersistentVolumeSelector labels in StatefulSet's volumeClaimTemplate

6/1/2017

So there is:

  • the StatefulSet to control several replicas of a Pod in an ordered manner.
  • the PersistentVolumeClaim to provide volume to a Pod.
  • the statefulset.spec.volumeClaimTemplate[] to bind the previous two together.
  • the PersistentVolumeSelector to control which PersistentVolume fulfills which PersistentVolumeClaim.

Suppose I have persistent volumes named pv0 and pv1, and a statefulset with 2 replicas called couchdb. Concretely, the statefulset is:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: couchdb
spec:
  ...
  replicas: 2
  template:
    ...
    spec:
      containers:
        - name: couchdb
          image: klaemo/couchdb:1.6
          volumeMounts:
            - name: db
              mountPath: /usr/local/var/lib/couchdb
      volumes:
        - name: db
          persistentVolumeClaim
            claimName: db
  volumeClaimTemplates:
    - metadata:
        name: db
      spec:
        ...

this StatefulSet generates two PersistentVolumeClaim named db-couchdb-0 and db-couchdb-1. The problem is that it is not guaranteed that pvc db-couchdb-0 will be always bound to pv0.

The question is: how do you ensure controlled binds for PersistentVolumeClaim managed by a StatefulSet controller?

I tried adding a volume selector like this:

selector:
  matchLabels:
    name: couchdb

to the statefulset.spec.volumeClaimTemplate[0].spec but the value of name doesn't get templated. Both claims will end up looking for a PersistentVolume labeled name=couchdb.

-- mmoya
kubernetes

Similar Questions

Kubernetes kube-proxy mechanism
Race condition on Pod restart
How to mount a Host folder in minikube VM
Rewrite all request targets in istio ingress controller

2 Answers

6/12/2017

If you are using dynamic provisioning, the answer is No you can not. Because a volume was dynamically provisioned is always deleted after release.

If not dynamic provisioning, you need to reclaim the pv manually.

Check the reclaiming section of k8s doc.

-- Ken Chen
Source: StackOverflow
1/5/2019

What you're looking for is a claimRef inside the persistent volume, which have the name and namespace of PVC, to which you want to bind your PV. Please have a look at the following jsons:

Pv-0.json

{
  "kind": "PersistentVolume",
  "apiVersion": "v1",
  "metadata": {
    "name": "pv-data-vol-0",
    "labels": {
      "type": "local"
    }
  },
  "spec": {
    "capacity": {
      "storage": "10Gi"
    },
    "accessModes": [
      "ReadWriteOnce"
    ],
    "storageClassName": "local-storage",
    "local": {
      "path": "/prafull/data/pv-0"
    },
    "claimRef": {
      "namespace": "default",
      "name": "data-test-sf-0"
    },
    "nodeAffinity": {
      "required": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {
                "key": "kubernetes.io/hostname",
                "operator": "In",
                "values": [
                  "ip-10-0-1-46.ec2.internal"
                ]
              }
            ]
          }
        ]
      }
    }
  }
}

Pv-1.json

{
  "kind": "PersistentVolume",
  "apiVersion": "v1",
  "metadata": {
    "name": "pv-data-vol-1",
    "labels": {
      "type": "local"
    }
  },
  "spec": {
    "capacity": {
      "storage": "10Gi"
    },
    "accessModes": [
      "ReadWriteOnce"
    ],
    "storageClassName": "local-storage",
    "local": {
      "path": "/prafull/data/pv-1"
    },
    "claimRef": {
      "namespace": "default",
      "name": "data-test-sf-1"
    },
    "nodeAffinity": {
      "required": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {
                "key": "kubernetes.io/hostname",
                "operator": "In",
                "values": [
                  "ip-10-0-1-46.ec2.internal"
                ]
              }
            ]
          }
        ]
      }
    }
  }
}

Statefulset.json

{
  "kind": "StatefulSet",
  "apiVersion": "apps/v1beta1",
  "metadata": {
    "name": "test-sf",
    "labels": {
      "state": "test-sf"
    }
  },
  "spec": {
    "replicas": 2,
    "template": {
      "metadata": {
        "labels": {
          "app": "test-sf"
        },
        "annotations": {
          "pod.alpha.kubernetes.io/initialized": "true"
        }
      }
      ...
      ...
    },
    "volumeClaimTemplates": [
      {
        "metadata": {
          "name": "data"
        },
        "spec": {
          "accessModes": [
            "ReadWriteOnce"
          ],
          "storageClassName": "local-storage",
          "resources": {
            "requests": {
              "storage": "10Gi"
            }
          }
        }
      }
    ]
  }
}

The volumeClaimTemplate will create two PVC test-sf-data-0 and test-sf-data-1. The two PV definition contains the claimRef section which has the namespace and PVC name on which PV should bind to. Please note that you have to provide the namespace as a mandatory because PV's are independent of namespace and there might be two PVC with same name on two different namespace. Hence, how does kubernetes controller manager will understand on which PVC, PV should bind, if we don't provide namespace name.

Hope this answers your question.

-- Prafull Ladha
Source: StackOverflow