I have setup a kubernetes cluster with three nodes. All nodes are Linux centos machines.
I need persistent volume to store data and I am trying to achive this.
I was following this tutorial. But it only covers a one node cluster. https://kubernetes.io/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/
Since, my cluster consist of three node, I could not use local path. Previous tutorial does not worked for me.
I need a network path and using NFS seems a reasonable solution to me. ( Is there any good alternative I would like to hear.)
Using NFS network mount contains two steps. First, Creating a persistent volume on a network path. Second, define this network path as a persistent volume and use it.
Second step pretty straight forward. Its is explained in kubernetes documentation and there is even sample yaml. documentation:https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistent-volumes example: https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/nfs-pv.yaml
First part also seems straight forward. Its explained in following document
/etc/exports
directory_to_share client(share_option1,...,share_optionN)
/etc/exports
/var/nfs/general 203.0.113.256(rw,sync,no_subtree_check)
/home 203.0.113.256(rw,sync,no_root_squash,no_subtree_check)But when you export a path as a NFS path you should make some configuration and give clients some rights. Basically you need client ip.
With kubernetes we use abstraction such as pods and we don't want to deal with real machines and theirs ip addresses. So, the problem startes here.
So, I don't want to give nodes ip to nfs server. (They might change in he first place.) There should be a better solution that all pods (in any node) should be able to connect to NFS network path.
Even allowing all ip without restriction or allowing an ip range might solve the issue. I would like to hear if there is such solution. But, I would also like to hear what is the best practice. How everybody else use NFS network path from kubernetes.
I could not find any solution yet. If you solved similar problem, please let me know how you solve it. Any documenatation on this issue will be good too. Thanks in advance!
You asked for best practices and from what I've found I think that the best option would be white-listing the IP addresses. Since you do not want to do that, there are also some workarounds answers posted on SO created by people who had similar issues with dynamic IP clients in NFS. You can find a link to deployment using glusterfs in the posted answers. If you want NFS with dynamic IP (because it can change), you can use DNS names instead of IP. If you need dynamic provisioning, use glusterfs.
I will add some information about the volumes as you asked. Might give you some light on the topic and help with the issue.
Since pods are ephemeral you need to move the volume outside the Pod - therefore making it independent from the Pods - so the volume would persist its state in case of Pod failure. Kubernetes supports several types of Volumes.
You could use NFS (more on NFS in the previous link) - so after the Pod is removed the volume gets unmounted, but it still exists. This is also not desired in your situation as the user needs to know the file system type and other details about the volume it will want to connect to. When you go to the examples in the documentation about NFS yaml files, you will see that their kind is defined as a Persistent Volume Claim.
This method is based on creating a series of abstractions that will allow a Node to connect to the Persistent Volume, but the user will not need any backend details, in addition, your node can connect to many providers.