I have a kubernetes cluster running on GCE.
I created a setup in which I have 2 pods glusterfs-server-1
and glusterfs-server-2
that are my gluster server.
The 2 glusterfsd
daemon correctly communicate and I am able to create replicated volumes, write files to them and see the files correctly replicated on both pods.
I also have 1 service called glusterfs-server
that automatically balances the traffic between my 2 glusterfs pods.
From inside another pod, I can issue mount -t glusterfs glusterfs-server:/myvolume /mnt/myvolume
and everything works perfectly.
Now, what I really want is being able to use the glusterfs
volume type inside my .yaml files when creating a container:
...truncated... spec: volumes: - name: myvolume glusterfs: endpoints: glusterfs-server path: myvolume ...truncated...
Unfortunately, this doesn't work. I was able to find out why it doesn't work:
When connecting directly to a kubernetes node, issuing a mount -t glusterfs glusterfs-server:/myvolume /mnt/myvolume
does not work, this is because from my node's perspective glusterfs-server
does not resolve to any IP address. (That is getent hosts glusterfs-server
returns nothing)
And also, due to how glusterfs works, even directly using the service's IP will fail as glusterfs will still eventually try to resolve the name glusterfs-server
(and fail).
Now, just for fun and to validate that this is the issue, I edited my node's resolv.conf
(by putting my kube-dns IP address and search domains) so that it would correctly resolve my pods and services ip addresses. I then was finally able to successfully issue mount -t glusterfs glusterfs-server:/myvolume /mnt/myvolume
on the node. I was then also able to create a pod using a glusterfs volume (using the PodSpec above).
Now, I'm fairly certain modifying my node's resolv.conf
is a terrible idea: kubernetes having the notion of namespaces, if 2 services in 2 different namespaces share the same name (say, glusterfs-service), a getent hosts glusterfs-service
would resolve to 2 different IPs living in 2 different namespaces.
So my question is:
What can I do for my node to be able to resolve my pods/services IP addresses?
You can modify resolv.conf
and use the full service names to avoid collisions. Usually are like this: service_name.default.svc.cluster.local
and service_name.kube-system.svc.cluster.local
or whatever namespace is named.