I have an Open Source project that I deployed in Kubernetes. It has a master, data, and query pod. The data pod is used to store the data. Unfortunately, when the data pod goes down it loses all its stored data as well. So I figured, I have to use a StatefulSet with PVC. I started looking at PVC samples. Suppose I create a PV:
kind: PersistentVolume
metadata:
name: task-pv-volume1
labels:
type: local
spec:
storageClassName: local
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data1"
And then I have a Claim and a POD to bind it. This is the sample Stateful Set:
kind: StatefulSet
metadata:
name: myweb
spec:
replicas: 2
...
volumeMounts:
- name: data
mountPath: /data
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: local
resources:
requests:
storage: 100Mi
Now when I run a sample job, I realize that only one of the 2 pods is active. The other pod fails saying:
Warning FailedScheduling <unknown> 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
which I believe is because I created only one PV. So I go and create a new PV,
path: "/mnt/data2"
and the pod is up and running.
In my prod setup, I want this to happen automatically. Which means I do not want to create these PVs for each replica. Is there a way to do this? Is this taken care of by Kube in some way?
Appreciate the help. If you have any further questions, please reply. I'll try my best to resolve it.
Currently the Kubernetes local storage provisioner is static and not dynamic. A dynamic provisioner would allow automatic spinning up of Volumes as mentioned in the post.
As the default Kubernetes local storage provisioner does not support this you will unfortuantely have to find another storage provisioner.
I would suggest looking at Ranchers local-path-provisioner: https://github.com/rancher/local-path-provisioner
It advertises in its overview
Local Path Provisioner provides a way for the Kubernetes users to utilize the local storage in each node. Based on the user configuration, the Local Path Provisioner will create hostPath based persistent volume on the node automatically. It utilizes the features introduced by Kubernetes Local Persistent Volume feature, but make it a simpler solution than the built-in local volume feature in Kubernetes.
Compare to built-in Local Persistent Volume feature in Kubernetes
Pros
Dynamic provisioning the volume using hostPath. Currently the Kubernetes Local Volume provisioner cannot do dynamic provisioning for the local volumes.
Cons
No support for the volume capacity limit currently. The capacity limit will be ignored for now.
You can install it by running
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
and then changing your storageClassName in your stateful set to local-path