I was trying to install a Cassandra stateful set on our k8s cluster. We want to mount an AWS EFS to all the Cassandra pods. And during configuring the volumes, I found we have 2 ways of declaring the volumes.
PersistentVolume
with the EFS-provisioner, and use volumeClaimTemplates
(I think it actually means persistentVolumeClaimTemplates, I don' know why they ignore the persistent prefix) in pod templates. Like this :apiVersion: v1
kind: Pod
metadata:
name: hello-openshift-nfs-pod
labels:
name: hello-openshift-nfs-pod
spec:
containers:
- name: hello-openshift-nfs-pod
image: openshift/hello-openshift
ports:
- name: web
containerPort: 80
volumeMounts:
- name: nfsvol
mountPath: /usr/share/nginx/html
securityContext:
supplementalGroups: [100003]
privileged: false
volumes:
- name: nfsvol
persistentVolumeClaim:
claimName: nfs-pvc
(code from a article)
apiVersion: v1
kind: Pod
metadata:
name: hello-openshift-nfs-pod
labels:
name: hello-openshift-nfs-pod
spec:
containers:
- name: hello-openshift-nfs-pod
image: openshift/hello-openshift
ports:
- name: web
containerPort: 80
volumeMounts:
- name: nfsvol
mountPath: /usr/share/nginx/html
securityContext:
supplementalGroups: [100003]
privileged: false
volumes:
- name: cassandra-shared-volume
nfs:
server: <EFS file system endpoint url>
path: "/cassandra/shared"
So for me, method 1 is just like wrap the nfs(EFS in this case) in a persistentVolume. I don't know what is the major benefits of method 1 though it is almost how every team use nfs. There is a short answer on efs-provisioner
FAQ : they says
I noticed the EFS gets mounted directly to the efs-provisioner container, can I do that for my apps? Yes you can but it's not recommended. You lose the reusability of the StorageClass to be able to dynamically provision new PersistentVolumes for new containers and pods.
This answer is fine but I am wondering if there is any implicts behind persistentVolume
than volume
alone. There is StatefulSet
doc saying :
The storage for a given Pod must either be provisioned by a PersistentVolume Provisioner based on the requested storage class, or pre-provisioned by an admin.
So I am wondering whether s StatefulSet must have a storage type of : PersistentVolume. The major conflicts is , unlike block storage, EFS itself is persistent and it's dynamic scaling ability is already handled by AWS and doesn't need persistentVolumeProvisioner
to create new resources.
I want to know is there any major benefit of method 1 or flaw in method 2 that I must use method 1 ? Thanks so much !
Use of SAN and network filesystems like EFS is not recommended for Cassandra in general, due to performance concerns. Depending on the workflows, you can get acceptable performance with ephemeral (instance) storage vs EBS, as explained here.
Adding the containers layer has not prevented those issues, at this moment we're conducting a similar Proof of Concept setting a write-intensive scenario, and we have been able to get some acceptable performance using i3 instances and ephemeral storage for the PersistentVolumes definition, but this is not a definitive response yet as we continue looking for the right configuration for our purposes.