I'm in the process of writing my own flexVolume driver on a 1.10.2-gke.3
cluster and ran into troubles getting it to work in attachable mode.
The driver works in non-attachable mode but then ReadWriteOnce
doesn't enforce mounting on only one node at a time which is a requirement for my driver and the ultimate goal I'm trying to achieve here.
I made the driver attachable by returning the capability in the init
call: {"status": "Success", "capabilities": {"attach": true}}
Then, when creating a pod that references the volume with this driver, only getvolumename
gets called. It doesn't really matter what I return here or if I implement the call at all. This is also the only invocation of the driver, i.e. never gets called for attach
etc.
The logs indicate that the volume is not registered as in use by the node:
kubelet[1910]: W0712 flexVolume driver a3cloud/glusterloop: using default GetVolumeName for volume a3cloud-system-test
kubelet[1910]: I0712 operationExecutor.VerifyControllerAttachedVolume started for volume "a3cloud-system-test" (UniqueName: "flexvolume-a3cloud/glusterloop/a3cloud-system-test") pod "test-56c867dd4d-75hwc" (UID: "4f633bb6-85b1-11e8-adb3-42010a9c0fc8")
kubelet[1910]: E0712 Operation for "\"flexvolume-a3cloud/glusterloop/a3cloud-system-test\"" failed. No retries permitted until 2018-07-12 08:57:24.360177039 +0000 UTC m=+64585.959480766 (durationBeforeRetry 2m2s). Error: "Volume has not been added to the list of VolumesInUse in the node's volume status for volume \"a3cloud-system-test\" (UniqueName: \"flexvolume-a3cloud/glusterloop/a3cloud-system-test\") pod \"test-56c867dd4d-75hwc\" (UID: \"4f633bb6-85b1-11e8-adb3-42010a9c0fc8\") "
According to this issue comment, as well as the the docs if the kubelet is started with --enable-controller-attach-detach
option the driver must also be installed on the master. Looking at the running processes, this option is not set, but may be a default in the meantime?
So my questions are:
Thanks!
In summary, GKE masters are managed by Google, so you can't access the master. It seems it's not possible to write a flexVolume driver with attachable support without having it to install on the master. However, for this question and the other question (mounting volume on only one node at a time) we recommend contacting the GKE team. They are available on Slack: slack.k8s.io.